1301.0 - Year Book Australia, 2001  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 25/01/2001   
   Page tools: Print Print Page

THE DEVELOPMENT OF OFFICIAL STATISTICS IN AUSTRALIA, AND SOME POSSIBLE FUTURE CHALLENGES


Bill McLennan

Bill McLennan CBE AM recently retired as Australian Statistician (1995-2000). He was previously Director of the Central Statistical Office and Head of the Government Statistical Service in the United Kingdom (1992-1994). Bill McLennan was also Chair (1994-1995) of the United Nations Statistical Commission, a subsidiary body of the Economic and Social Council of the United Nations.

footnote: While taking full responsibility for the article's content, I am grateful for the many helpful comments and suggestions from Dennis Trewin, Tim Skinner and Rob Edwards of the ABS, and Brian Pink (formerly of the ABS, now heading Statistics New Zealand).



Introduction

This article briefly describes the development of official statistics in Australia,1 and highlights and compares the situation 100 years ago with today. It also presents the author’s views on some of the challenges that could face Australian statisticians in the years ahead.


Colonialisation to Federation

Initially, the colonialisation of Australia, which commenced in 1787, was seen as an economic means of disposing of felons. The British Government was insistent that comprehensive accounting records were kept, and that reports were prepared on social and legal matters, to judge whether this experiment was a success. The first Governor-in-Chief, Arthur Phillip, was implicitly instructed to collect and collate information in numerical form, which he duly did in a report to his masters each year. The same story applied to the other colonies as they were established.

The first major change occurred in 1815 when, following an overhaul of colonial administration, the systematic collection of a wide range of data was commenced, primarily it seems for the UK Colonial Office. In 1822 an annual reporting system was set up, called the Blue Books, and their emergence probably reflected the new imperial situation following the loss of the American colonies and the end of the Napoleonic wars. While the Governor was formally responsible for their production, the Colonial Secretary, a public servant, did the actual statistical collating. The topics in the Blue Books reflected the British preoccupation with the cost of the colonies, and the Blue Books appear to have been regarded rather as documents for the guidance of the administration in the Colonial Office than as a contribution to a scheme of statistics for Australia, let alone internationally. However, local influences increasingly affected the character of these books, and the practice developed of retaining copies in the colonies for local use. As a result, it is in the work of preparing them that we find the germ of the development of Australian statistics.

A most significant statistical occurrence around this time was the conduct of the first formal census, which was held in the colony of New South Wales in 1828. Prior to this the counting of the population had been by musters. Thereafter, Censuses were held in NSW and the other colonies at regular intervals, with the form and timing of the Censuses being decided in the colonies.

Self-government in the colonies occurred progressively from 1855. Then it was entirely for the colonies to decide on the range and quality of their statistical records. In all the colonies after a transition period, the Blue Books were transformed into the Statistical Registers of the second half of the nineteenth century, and although there were marked similarities, the responses of the colonies inevitably were different. In turn these Statistical Registers were transformed into Year Books with analysis of and comments on the statistics.

Using the Victorian colony as an example, because its experiences mirror what happened in the other colonies, the first step was the establishment in early 1855 of a Registrar of Births and Deaths and the setting up of an elaborate and uniform system to record vital events and derive comprehensive statistics from them. Within the office, a Statistical Branch was also used to collect agricultural statistics and produce annual statistics for Victoria. However, there is reason to believe that the Registrar-General was inclined to look upon the compilation of the annual Statistical Registers as an onerous addition to his legitimate work, and that therefore no serious attempts were made to improve the publications. As might be expected, the lack of uniformity in the coverage and the presentation of statistics in these annual volumes across the colonies was keenly felt. Indeed:
    "The Government of Victoria expressed a hope ... to the effect that the three colonies should not only unite in regard to the enumeration of the people, but to recast and assimilate, in concert, all “blue books” and other statistics, on a scientific and practical basis, [and that this] would meet with the concurrence of the Government of New South Wales and South Australia." 2

In 1874 the Statistical Branch was separated from the Registrar-General’s office and established as a separate organisation in the Department of the Chief Secretary, with its head being called the Government Statist, the first such appointment in the colonies.

The next major step forward was the publication of an annual summary of Victorian statistics, containing not only bare tabulations, but also analyses of the figures; the Victorian Year Book was to fill that role. As the Government Statist at that time said:
    "It will be my endeavour in this succession of volumes to record facts with correctness and impartiality; to comment on them only so far as may be necessary as to elucidate them properly; to set up no theories except such as may be fairly deducible from the materials before me; and, in drawing inferences, to exercise perfect fairness to all sections of the community." 3

This quote shows that the principles of a sound, open and objective statistical system were established in Australia many years ago.

Continuing with the Victorian example, the responsibility for conducting the 1854 Census was given to the newly created Office of the Registrar-General. Along with attempts to produce uniform annual statistics in the second half of the 1850s, discussions and negotiations began about holding the Census in 1861 in all of the Australian colonies on the same day as it was held in Great Britain and Ireland. In the event most colonies did so and censuses were then conducted every ten years.

At this stage of development, official statistics of the Australian colonies were certainly of high international quality, both in content and presentation. This may seem surprising, but it has been said that:
    "To a considerable extent the achievement was, for a number of reasons, a legacy of British colonial rule. First, the colonies had been required to produce official statistics on an annual basis.... Second, the statistics had to be of a range and quality to satisfy the British authorities, who required them for efficient administration. Third, the statistics had to be brought together by a single officer, the local Colonial Secretary, who took some final responsibility for their accuracy and their presentation; there was therefore a central statistical authority and this contrasted markedly with the British position. Finally, the authority was required to present all the relevant statistics of the colony in a single volume - the Blue Book. As an offshoot of these developments, it was natural for the colonies to begin the production of a consolidated volume of annual statistics for their own use." 4

Things, however, were not quite as rosy as that quote might indicate. Due to many reasons, not the least being the inequality in the funding of the various statistical offices of the colonies, the quality and the scope of the statistical work emerging from the colonies varied enormously. Attempts were made to overcome this, especially by including in the Victorian Year Book and The Seven Colonies of Australia published by New South Wales a large amount of information relating to all the colonies. Knibbs, the first Commonwealth Statistician, put the issue rather forcefully as follows:
    "Notwithstanding this early recognition that the Statistics of Australia should develop a uniform plan, the autonomy of each State led to divergencies of domestic policy and practice. These divergencies tended also to manifest themselves in the statistical technique, as well as in the facts collated. Even where there seemed to be unity of action, or identity in the data to be collected, the unity and identity were often more apparent than real. The comparative studies made by each Statistician revealed with more and more clearness, in proportion as they were thorough, the grave lack of uniformity in the statistical data and methods of the several States, however excellent these may have been considered alone." 5

Several conferences of the Statisticians of the various colonies were held from 1861 to deal with the matters where coordination was most urgently required. Beside the exchange of ideas, the main result was to maintain a certain degree of rapport between the Statisticians of the colonies, rather than to bring about any very fundamental advance in statistical method. Agreement was reached on what was to be included in 'Australian Statistics' and considerable agreement was reached on the conduct of several Population Censuses. Knibbs did say later that there was “still much to be achieved”. 6

At the time of Federation, there was a Government Statistician, or Statist, in all colonies except in Queensland and Western Australia where they were appointed in 1904 and 1918 respectively. The range of statistical data the bureaux were dealing with at that time was considerable, and was approximately as follows:
    • statistics collected entirely by the bureaux included agriculture, dairy farming, live stock, manufactories, municipal administration, hospitals, asylums. etc.;
    • statistics compiled in the bureaux from data collected by other public departments included population, births, deaths and marriages, banks, life assurance, trade, shipping, and criminal justice; and
    • statistics collected and compiled by other agencies and adapted for publication by the bureaux included public finance, railways and tramways, posts, telegraphs and telephones, land settlement, meteorology, mining production, water conservation and irrigation, civil justice, and public instruction, scientific societies, museums etc.


Federation to integration

Federation of the colonies into the Commonwealth of Australia on the 1 January 1901 had many implications for official statistics in Australia. The first major task was the 1901 Census, and from the outset it was clear that generally accepted population figures would be essential as a basis for apportioning to and for the States. This indeed was an important issue with respect to the status of official statistics in the new country, and the Statistical Conference held in February 1900 addressed these matters.

It is interesting to note that almost immediately there was discussion among the newly created States of Australia, and indeed New Zealand, about the need for uniformity in the preparation of statistical returns. This was an urgent matter in the view of the Commonwealth, so that its administration could be well founded on a good statistical base, and one which would allow international comparisons. A conference of the relevant Statisticians discussed this first in January 1902, but while there was agreement on the need for uniformity, as there had been for many years, no concrete agreement was reached about how to proceed. This was to be a continuing theme for many years, including discussions between Commonwealth and State Ministers.

After conferences with the State Governments, the Commonwealth Government decided to exercise its power under the Constitution to place the responsibility for taking Censuses of the Population and the compilation and publication of statistics with the Commonwealth. In 1905, the new Census and Statistics Act was approved by Federal Parliament. It dealt among other things with the appointment and powers of the Statistician, arrangements with the States for the collection of data and responsibilities for the taking of Censuses of population. The second reading speech of the Minister for Home Affairs, when he introduced the Census and Statistics Bill into Parliament, made several interesting points. He said: "The object of the Bill is to enable the Commonwealth to establish a central bureau of statistics in order that it may ... publish certain statistics having reference to the affairs of Australia as a whole". The reason for a centralised Bureau was given as a need to "bring into line the statistics of the States for the purpose of comparison, and to lay down a uniform method for the collection of statistics". In addition, "the central department will collect all information in regard to subjects specially controlled by the Commonwealth, such as imports and exports, trade, and commerce generally including inter-State transactions, navigation and shipping, postal, defence and other matters". He also said: "we could establish a central Commonwealth bureau and enter into negotiations with the various States with a view to utilising their departments to the fullest possible extent".

The first Year Book 7 reported, in relation to the organisation of the Commonwealth Bureau, that

      "foreign Governments were asked - (a) to furnish their statistical publications, including such back numbers as could be spared; and (b) to enter into a general agreement for exchange of publications. It would be impossible to speak too highly of the generous response which has been made to this request, a request to which a young country like Australia can make adequate return only in the somewhat distant future".

It is good to record that some significant return has been made to the international statistical community generally over the years since then.

Practically the first matter of importance to be considered after the passing of the Census and Statistics Act was the question of the relations between the Commonwealth Bureau and the existing State Bureaux. Knibbs’ comments about this pivotal issue are revealing:
      "Two methods of procedure were open to the Federal Government. The first was the complete unification of all statistical organisations in Australia. If this had been adopted the Commonwealth would have controlled all statistical work, and would have been represented in each State by a Branch office which would have undertaken the collection and first tabulation of statistical data under the direction of the central bureau. A second method was to preserve the internal independence of the State Bureaux, and to arrange for them to furnish the Federal Bureau with data compiled according to a system agreed upon. The Federal Government chose the second method as being, at present, and in view of all circumstances, more suitable to the actual condition of Australian Statistics, and it was thereupon resolved to hold a conference of Statisticians which should discuss the arrangements to be made in order to satisfy the requirements of the State Governments as well as those of the Federal Government." 8
The upshot of this was that there was a Federal Bureau, as an office within a Commonwealth department, and State Bureaux, also as offices within State departments, operating independently, but with respect to Australian statistics cooperating with the Federal Bureau. The regular meetings of the Chief Statisticians of the Federal and State Bureaux, termed the Conference of Statisticians, was the mechanism used to coordinate activities aimed at producing uniform Australian statistics. This process worked to some extent but was slow, tedious and not very effective.

The first conference after the establishment of the Commonwealth Bureau of Census and Statistics occurred late in 1906. At that meeting a set of statistical forms were approved that the State Statisticians agreed to compile and furnish to the Commonwealth about their respective States for the purposes of compiling statistics for Australia. It is not surprising that these arrangements were fraught with difficulties. Indeed in 1908 the then Commonwealth Statistician advised his Minister that the Commonwealth Bureau “is at the mercy of the slowest and least efficient State Bureau for the completion of practically the whole of its statistics”. This crippling dependence was obviously irksome. “Unless more strenuous efforts are made by the States to supply the Commonwealth with statistical information it will become necessary for the central authority to obtain statistical information directly instead of through the State Statisticians.”

In addition, it was soon found necessary for the Commonwealth Bureau to undertake original compilations, and develop the scope of its work beyond the mere summarisation and analysis of returns furnished by the State Bureaux. The Commonwealth Bureau moved into the direct compilation of commerce and shipping, vital, industrial, employment, wages and prices statistics. It also put a lot of effort into the publication of the Official Year Book of the Commonwealth of Australia, the first issue of which appeared in 1908.

Indeed this matter of coordination, or more specifically the lack of it, was discussed by Ministers of the Commonwealth and the States, and in 1916 a motion was passed in favour of amalgamating the statistical bureaux. This did not come to pass, in part one suspects because the State Statisticians remained unwilling to surrender the autonomy that they and their predecessors had enjoyed for so long. Indeed, many unsuccessful attempts were made over the next 30 years to have significant organisational changes implemented. A major one, which was partly successful, occurred at the 1923 Premier’s Conference when the Prime Minister proposed that the State statistical services be transferred to the Commonwealth. At the time Tasmania alone agreed, and this transfer was effected in 1924. A direct result was the significant influence statisticians from Tasmania then had on the Commonwealth Office by the transfer and promotion of staff to senior positions, including as Commonwealth Statistician, in the Commonwealth Bureau in Canberra.

The rapport the Commonwealth Bureau had with Australians in conducting the Population Census is well shown by the following poem: 9
      My happy home, one week ago, my bungalow, “The Nest”,
      was bounded north by Paradise, the place of perfect rest.
      Now all is gloom where gaiety and merriment erst reigned,
      relations with my better half are most distinctly strained.

      On Sunday night, I gaily took my fountain pen in hand,
      and started to enumerate for Knibbs our little band.
      At question 1, I paused, and to my nice lady said:
      “At last I get a chance to write myself down as “The Head”.

      No better wife than mine has shared the lot of mortal man,
      she is a perfect helpmate, in accordance with the plan,
      but on one point that gentle dame I’ve never dared to cross;
      she has been, ever since we wed, undoubtedly the “The Boss”.

      And when I wrote that I “John Smith, Bricklayer,” was the head,
      what hit me hard was what she looked—for not a word she said.
      Her name is Marian, she says, but as an honest man,
      I wrote it from the record of her birth as “Mary Anne”.

      She looked her wrath - a wiser man had better know his mate.
      I wrote her age - per record still - “next birthday 48”.
      Then Marian broke loose, and said unpleasant things to me.
      I told her all the questions must be answered truthfully.

      Her speech was brief and to the point. She rose and left the room.
      Our happy home is now about as joyous as a tomb.
      We do not speak, as we pass by. At least she will not speak,
      and this condition will endure for all the wretched week,
      because I did as Knibbs decreed and, acting in good sooth,
      wrote for him “confidentially” some brief domestic truth.

It is pleasing to note that this spirit of cooperation continues today.

Returning to the question of the overall statistical service, there seems little doubt that all the State Governments used their own statistical services as a significant input to their policy and planning activities, just as they had in the colonial days. This rather intense interest in statistics waned over the years as the power of the Commonwealth increased. Finally, the advent of the Second World War forced the issue in that the governments of Australia had to think collectively.10 Indeed certain fields of statistics, such as manpower, physical controls on the economy (with the market economy being suspended), production and use of materials, were recognised as vital to the Federal Government for the prosecution of the war. The Federal Government also took over the collection of taxes. The corollary was that the State Governments needed statistics less and less. This trend continued after the war, when the focus was on post-war reconstruction. The Federal Government’s commitment to a full employment policy, which it embodied in a White Paper in 1945, had great significance for the scope of the Commonwealth Bureau’s role. At that time statistics of manpower, production, stocks, trade, economic indicators and national accounts all became essential instruments for the Federal Government, with the Commonwealth Bureau being in control of their production. Also the buoyant economic conditions after the war did not encourage the State Governments’ interest in monitoring the course of the economy.

As it happened, fate may have played a significant hand. It turned out that an acting appointment, in the late 1930s and straight after the war, from the New South Wales Bureau to the office of Commonwealth Statistician was a trigger to progress the integration of State and Federal Statistical Bureaux. A further significant boost was the strong endorsement of the Prime Minister and the then Premier of New South Wales in the late 1940s to 'closer coordination'. In fact, the Commonwealth Bureau staff in New South Wales and the State Department staff were housed together. Finally, a series of separate agreements, between the Commonwealth and the States in 1957 and 1958, created an integrated statistical service where the State and Federal Bureaux became one organisation operated by the Commonwealth, which was responsible for serving both Commonwealth and State Governments' statistical needs. Each State Bureau became an office of the Commonwealth Bureau under the immediate direction of the existing State Statistician, who then held office under both Commonwealth and State legislation. No State would be required to surrender its sovereign powers in the field of statistics, but they agreed to exercise them in a special way through an integrated service. The reality was that the State Governments generally had shown little interest in, or expenditure on, statistics in the late 1940s and 1950s, and so what they handed over was a depleted statistical compiling capacity with limited methodological or interpretive skills.

The integrated statistical service had a long gestation period with a difficult birth; its growth toward maturity would have difficulties as well.


Integration to the new millennium

For about ten years thereafter the operation of the integrated statistical service changed very little from when it was a collection of separate and independent offices. In one sense this is not surprising given the history of the organisation and its staffing. However, there were many forces operating to influence how this statistical service would develop. Some of the important ones were very significant increases in the demands for statistics, the need for efficient and long-term management, the impact of technology and the requirement for international comparability. In fact, all these forces are so closely intertwined and interlinked that it is impossible to trace the influence of each separately. Rather, some key aspects will be highlighted, roughly in the order in which they occurred.

Over the next 20 years, the demand for official statistics increased significantly due to the greater use of statistics in planning and research, not only by the public sector of the economy but also by private enterprise. Existing collections were enlarged and new ones instituted in the various economic and social fields. In addition, the State-based factory and agricultural censuses were replaced by national collections, which was a significant step towards standardisation. Also, towards the end of the 1960s it became obvious that there were shortcomings in many of the important economic statistics. As many of these series had a common origin, the idea was put forward of ‘integrating’ all the economic censuses and surveys. This became a nation-wide project aimed at ensuring that the collections for each industry would fit together without overlap, duplication or omission in coverage, and produce a range of economic data according to a common system of concepts. To do this it was necessary to standardise all the data collected and to prepare a register of businesses operating in Australia.11 This process impacted strongly in all offices and gave another thrust to the move to a more centrally planned and managed approach. It is worth noting that the impact of the integrated economic collections, and the growth in the demand for statistics, was significant on both the Federal and State organisations. Again using the example of Victoria, at the time of integration in 1958 there were about 100 people employed in the Victorian Office, but by 1977 there were nearly 500 (today there are 350).

To meet this new demand, staffing not only had to be increased but the right type of staff had to be employed. After integration, this was an important issue as most staff in the State Offices were male and normally ex-service, whose strength was in keeping an operation running but not usually in initiating change. The most significant step taken to address these matters was the recruitment of many graduate staff, and of particular importance here was the statistics cadetship scheme. In this scheme, which still operates today, the Bureau employed young people before they finished their degrees.12 The scheme was remarkably successful, having a strong and continuing impact in both Central and State Offices. Many graduates of the scheme went on to senior level positions in the Bureau including as Australian Statistician, in other government departments including as Permanent Secretaries, and in the private sector.

In another sense, the slow change at the start of the integrated statistical service was not surprising, given the size of Australia and the difficulties that then existed with communication. However, technology became a driving force for change. The large volume of statistical information being collected could no longer be handled efficiently by manual methods, and after the Second World War various types of mechanical processing were implemented. In the early 1960s the Commonwealth Statistician established a computer network in all State Offices, which from the beginning was closely managed from the centre. The growing use of technology thus became a strong impetus for a more coordinated and centrally managed statistical system. (As an example of the interlinking of these forces for change, it is interesting to note that the development of this network required many skilled computing experts who were not readily available in Australia. The solution was twofold. First, to undertake extensive training of staff in the skills required, and these courses predated any undergraduate computing courses offered by Australian universities. Second, to recruit many such people from the United Kingdom. They served the Bureau very well, and eventually had a significant impact on computing elsewhere in Australia.)

Another important change was the introduction in the mid 1960s of a capacity to conduct interview surveys at households across Australia. It was initiated by the Commonwealth Government specifically to run a labour force survey. This time-critical collection, which employed a large number of part-time interviewers across Australia, was centrally planned and centrally managed. This was another forerunner of the changes that would be made in the management of the integrated service, with Central Office senior management having more direct involvement in the activities of the State Offices. It also provided the basis on which the ABS was able, in the years to come, to build its large program of labour and social surveys, collectively called the Population Surveys.

A further important factor which influenced the way the integrated service operated was the upsurge in the use of sampling and other methodological techniques, which were used to help meet the demand for more and different statistics within reasonable costs. People with these skills were, and still are, scarce resources, and it was natural, therefore, for them to be generally located in the centre. As their work encompassed the whole of the integrated service, there is no doubt that they were, and still are, a strong unifying force. Although the cadetship scheme was, and still is, a great source for people with these skills, there was a need to recruit and to mount special recruitment campaigns.

By the early 1970s, although the Bureau’s statistical output had expanded considerably and had a more contemporary flavour, the management of the statistical service was still firmly based on the conduct of annual Conferences of Statisticians, but with Central Office playing a more decisive role, due in part to the central management of many of the new initiatives and services. Significant changes had taken place in the staffing of the organisation, especially in Central Office.

At around this time, the Federal Government began to ask more of the statistical service. It believed that it was important for the quality and consistency of governmental and parliamentary decisions, and for the community's capacity to evaluate them, that official data systems, particularly in closely interrelated areas of social and economic policy, be mutually compatible and reflect the dynamic needs of the Australian economy and community. To this effect, the Government commissioned an independent inquiry, the 'Committee on the Integration of Data Systems', which reported in 1974. Among other things, the Committee concluded that coordination and compatibility were more likely to be achieved under a centralised than a decentralised form of statistical organisation, and that if any such central statistical authority were established it should be user-oriented, serve the government and the community as a whole and be policy neutral, and that its output should reflect an appropriate balance between different fields of economic and social statistics and between short, medium and long-term needs.

This led to the establishment of the Australian Bureau of Statistics in 1975 as an independent statutory authority, with the Australian Statistician given a secure and fixed-term appointment. (Previously, the ABS was in effect an agency of the Treasury Department.) The Statistician was made responsible for the control of the operations of the Bureau, and its functions were:
    • to constitute the central statistical authority for the Australian Government and, by arrangements with the Governments of the States, provide statistical services for those Governments;
    • to collect, compile, analyse and disseminate statistics and related information;
    • to coordinate, and provide advice on, statistical activity at the Federal level (although by invitation it is sometimes done at the State level as well); and
    • to liaise internationally with respect to statistics.

Further, an Australian Statistics Advisory Council was established, with its members representative of users and suppliers of statistics, to advise on long-term statistical priorities and strategic issues.

For the first time Australia's statistical agency was organisationally independent of any department of State. Further, the Statistician was given the powers of a Departmental Permanent Head in respect of the Public Service Act. Perhaps, at this stage, it could be considered that the integrated statistical service had just reached adulthood.

A major step in its maturing process, which came soon thereafter, was the development and adoption of a coordinated and comprehensive planning and management system for the Bureau. The aim was for the first time to manage the ABS strategically by stimulating long-range thinking, by responding to the changing environment, and by reassessing what the Bureau did and how. It was based on developing a corporate plan, containing a mission statement, to give the medium term perspective, and setting in place a rolling three-year forward work program which was updated annually to ensure that the Bureau used its resources efficiently and effectively. At the same time matrix management was implemented, with statistical and services program managers in the Central (i.e. Canberra) Office being responsible in a planning and strategic sense for all activities in their programs across all offices, while the day-to-day operational responsibilities lay with the senior managers in each State Office. This once again reinforced the holistic approach being taken to the management of the statistical service. It really heralded the first time the integrated statistical service operated as a whole rather than as the sum of its parts. The ABS is still successfully managed this way today. (It is interesting to note that around this time, perhaps as a direct result of these changes to a centrally managed statistical service, the Annual Conference of Statisticians ceased and was replaced by biannual meetings, of the senior staff of all Offices of the Bureau, which were more management focused.)

The maturing of the computer services and infrastructure in the ABS brought on other changes that had significant impacts. In the absence of well-developed computing facilities in the State government agencies, the State Offices had undertaken a significant proportion of the processing work associated with State administrative systems, such as crime, education, traffic accidents, industrial accidents, hospital morbidity etc. As plans were being made to move from clerical to computer possessing, the question was asked whether the ABS should be responsible for this processing work. It was decided that the interest of the ABS was in the summary data coming from the administrative systems, not in the data entry and the associated computer processing work, and that work should be 'returned' to the owner of the administrative systems, i.e. the State agencies running these systems. This decision was not popular with the States, as it imposed a load on them that the Commonwealth had been carrying. Importantly, this change did allow more ABS resources to be channelled into other social statistics work, which the ABS was best placed to do: specifically to use its capacity to efficiently conduct household surveys. Indeed, this change was the first time the ABS went beyond relying on the Population Census and administrative by-product data for most of its social statistics. The buildup of the Population Surveys started with the poverty surveys in the early 1970s conducted for the Henderson Poverty Enquiry, and with health surveys soon thereafter.

At around this time, a fundamental change occurred in the way the ABS viewed itself. It acknowledged that its success, or otherwise, was not judged solely on its output, but rather on how well its statistics are used by governments and the community. In doing so the ABS accepted that it should take positive steps to direct and encourage the flow of statistics, both published and unpublished, to users, i.e. to 'market' its statistics. Indeed this thrust is encapsulated in the ABS mission:
      "We assist and encourage informed decision making, research and discussion within governments and the community, by providing a high quality, objective and responsive national statistical service."

Marketing is an ongoing activity that covers all means of dissemination, and includes all types of contacts with users. Indeed today the opportunities offered by technological change, including the ability to store and access vast amounts of data easily and efficiently as well as disseminate them, place great importance on this function being undertaken well. Indeed, the ABS has expanded its publication program very significantly both for paper publications, now exceeding five per working day, and electronic dissemination, where publications and other publishable statistical data can be accessed electronically over the Internet. In addition, a big effort has been made to implement user friendly and consistent publication standards.

A very important change occurred in 1981 when the Census and Statistics Act 1905 was significantly amended. The impetus came from the Australian Law Reform Commission’s report Privacy and the Census, which recommended that there was a need to reaffirm and clarify the importance of confidentiality in the statistics legislation. However, the amendments achieved many other things; the most important was perhaps that they put into law the power given to the Statistician in 1975 to “control the operation of the Bureau”. Specifically, they removed all references to the Minister in administrative areas, and said that the Statistician would determine what statistics would be collected, when and how often. They also provided that the Statistician could direct a person to supply information requested, and institute legal action if necessary which carried possible substantial penalties for non-observance. On the other side of the operations, they placed an obligation on the Statistician to compile and analyse the information collected and to publish and disseminate it. The changes were also aimed at providing a better and different service to users of statistics, particularly by enabling unidentified and unidentifiable unit record information to be released under very tight and specific conditions; this service has proved to be most effective, popular and well appreciated by most users of data. There is no doubt that by this time the integrated statistical service had matured a little.

A major change in how economic statistics were to be collected was agreed during the mid 1980s, with much more reliance to be placed on using income tax information, both as a better source of information to update the ABS business register and as a direct source of data, particularly for small businesses. This decision offered two advantages: first, improvements to the statistical output, especially for small domains, particularly small area information, and second, a reduction in the load placed on small businesses. At the same time it was decided to place greater emphasis on economy-wide surveys and to directly link, and confront, the national accounts with the various economic censuses and surveys. After a lot of complex work, this strategy is coming into place and is being aided quite significantly by the current implementation of the goods and services tax. Both users and suppliers of data should benefit considerably from these major changes.

In the late 1980s, the ABS further extended the household survey program, committing to regular 'core' surveys in high priority areas (income, health, disability and the aged, and childcare). By the late 1990s, the program had been further expanded, mainly by the injection of user funding for specific surveys, but demand continued to grow and regularly exceeded capacity. In 2000, the ABS has further extended the program to meet some high priority gaps (Indigenous statistics, more frequent health surveys, general social surveys), to respond to contemporary needs and to enable most areas of social concern to be covered from within the ABS budget.

The most significant organisational change in the statistical service undertaken recently has been the centralising in the various State Offices of the collections for specific subject matter fields. Up until then, all State Offices collected and collated statistics for most collections run by the ABS, i.e. it still mirrored the pre-integration approach. No longer is this the case. For example, all agriculture statistics are collected and processed by the Tasmanian Office. This is a form of centralisation, but it is geographically dispersed, with each State bearing responsibility for a set of specific subject matter fields. (This solution, I expect, should probably be viewed as unique to Australian conditions as it is a product of our history.) This change has achieved large efficiency gains, even over and above the reduced management overheads, because of a stronger focus on operational process improvement. Effectiveness has also improved, perhaps due to re-engineering which had to be done to implement the change, and there have been quality improvements through the impact of standardisation.

Another big difference from the past in the statistical service today is the emphasis that is now placed on the analysis and interpretation of the statistical output. This has many facets, ranging from analysing the data in each statistical release and explaining their implications to users, to being involved in the building of models of the economy. Included in these activities are the calculation and publication of seasonally adjusted estimates and the provision of 'trend' information so that data can be better understood; initially there was controversy surrounding their publication, but both are now considered helpful by users. The ABS also publishes some well accepted, comprehensive analytic publications like the quarterly Australian Economic Indicators and the annual Australian Social Trends. Some bold steps were taken in the early 1970s to adjust the population estimates for Australia to take account of the known, and differential, undercount coming from the Census. (The way this decision was, and continues to be, well accepted by all users, shows how good the ABS’s integrity is in the eyes of users; this has been reinforced by the strident reaction statistical agencies in some other countries have faced when they have suggested a similar solution.) More recently, some important work has commenced on measuring productivity in parts of the services sector, and with a reasonable level of success.

Over the last 40 years, the Bureau has been increasingly active in the international scene; there are several reasons for this. First, statistical standards, which are important within a country so that statistical series can be easily and accurately related to each other, are also essential so that valid international comparisons can be made, a requirement which has become increasingly important over the years. For many years these standards have been put together and sanctioned at the international level. The major arena is based on the United Nations, and within that on the UN Statistical Commission (UNSC), a representative group of official statisticians from a range of countries, which is a subsidiary body of the UN Economic and Social Council. The role of UNSC should be paramount in terms of standard setting, but in reality it acts as an advisory body to the UN Statistical Division, the statistical arm of the UN Secretary-General. Liaison is thus still necessary with the statistical arms of other UN bodies, such as ILO, FAO, WHO and UNESCO, and with international agencies such as the World Bank, the International Monetary Fund and Eurostat, the statistical arm of the European Union. ABS staff have been very active and significant contributors to all these agencies, and within the UN structure they have played leading and determining roles at the UNSC and at the Committee on Statistics of the Economic and Social Commission for Asia and the Pacific (ESCAP), the regional arm of the UN. Second, the Bureau has always tried to be a 'good international citizen', particularly within the Asia-Pacific region. It has adopted an open door policy by welcoming delegations from the statistical agencies of other countries, and many countries take advantage of it each year. It has also been active in providing assistance to statistical agencies that are not so developed, in the form of consultancies and the provision of short-term experts; the philosophy has always been to help and advise but not to do. These efforts have often been lauded in international forums. The Bureau can honestly claim that it has made at least “adequate return” as promised by the Commonwealth Statistician in 1904, and it can be expected to add interest to this return in the years ahead.

An important point not made before is the long-term investment by the ABS in recruiting, training and developing its staff. There are many facets to this, but three are worth highlighting. First, the employment of young, promising undergraduates, and paying them to complete degrees before commencing work in the ABS, has been very successful in terms of long-term tenure. Second, there is a significant expenditure per annum of at least 5% of budget on staff development. Third, there is a policy to ensure that all senior staff have had a broad range of work experiences, across various subject matter and service areas and in regional offices as well as Central Office. The result has been high quality staffing, and staffing at senior levels with the experience and training to foster and maintain a strong corporate and strategic view, which is the mode of operation of the senior management group of the Bureau. The success of this approach is shown by the large number of ex-ABS staff who have worked, or are working, in senior positions in the official statistical agencies of other countries.

The effective use of technology has had a long impact on ABS operations, in addition to the role it played as a driving force for change in the early 1960s. Indeed, the ABS has built a lot of its productivity improvements around its strategic technology decisions. Five issues are relevant here:
    • involvement of the most senior group of ABS management in strategic technology planning since the early 1960s;
    • adherence to a corporate governance model for IT that has resulted in a very high degree of standardisation of our hardware and software infrastructure across all of our offices around Australia;
    • significant and ongoing technology-related skills training for all ABS staff to ensure effective application of technology in business processes;
    • strong security policy, procedures and practices for our IT environment; and
    • most recently, the development of a single logical repository, known as the ABS Data Base (ABSDB), for the documentation, storage and management of data and associated information describing and locating them for dissemination and reference purposes.

More recently, the advent of desktop computing, more efficient networks, effective workplace tools (such as Lotus Notes), the Internet and the use of video conferencing are rapidly changing the dynamics of this large and dispersed organisation.

I should point out that although Australia has, in the main, a centralised statistical system, not all statistical work is done in the ABS. A reasonable amount of statistical activity is undertaken in Federal and State government departments, but generally speaking the ABS is across most of this activity. For example, the ABS works closely with the Australian Institute of Health and Welfare, which has a significant statistical role with respect to statistics produced from the administrative systems of governments in these fields. Another model of cooperation has been the establishment of national statistics centres in the fields of crime and justice, culture and leisure, Aboriginal and Torres Strait Islander statistics and, quite recently, education and training. Here the ABS has formed partnerships with the key players, including part funding, to develop statistics in these fields, particularly from administrative systems

At the millennium, the integrated statistical service has matured a lot, and serves the governments and the people of Australia very well. It produces a massive amount of output, which is both quite diverse and detailed, and its staffing levels have grown to over 3,000. The service is well managed with a firm eye being kept on future requirements and demands, and with a positive approach to staff development. It has an enviable record in maintaining the secrecy of the data collected, and as a result receives very strong support from respondents, enjoying cooperation rates better than elsewhere in the world. Its use of technology for the production of statistics, for the dissemination of data and other information, and for management, is at the leading edge. It has an outstanding reputation for both its performance and its international contribution, from inside Australia and world-wide.


The millennium onwards

From this strong and stable base, the ABS is well situated to develop further to respond to the changes that will happen in society and government in Australia and internationally. The basic role of the ABS is unlikely to change, although how it does things and what it does may change significantly. The providers and users of statistics will still expect that the best possible and most appropriate set of statistics will be available to governments and the community, and that these will be provided without fear or favour. The Australian Statistician will continue to carry significant personal and community responsibility; the position is in effect the managing director/chief executive officer of a large public company, charged with showing a return (in terms of its statistical output) to its owners and shareholders, the governments and the general public of Australia. Many forces might come to bear on this process, including changes in the focuses of government, in community attitudes, in the availability and use of technology, and in international requirements.

The recently adopted ABS Corporate Plan, 2000 has as its first objective “an expanded and improved national statistical service”. The aim here is for the ABS to make better use of data available from all sources, whether from government or private sector administrative or transactional data sources, in providing official statistics to users. This is a formal recognition that the ABS cannot collect all data, but what it can do is utilise its skills in managing and disseminating data to ensure that users have the best national statistical service possible. No doubt this is a step in the right direction, even though it may only be the first one.

What has already happened, and it will continue, is that official statisticians are no longer the only ones looked to for credible data for decision making.13 This will bring with it no doubt, if it has not already happened in some countries like the USA, the loss of the unquestioned authority of official statisticians; thankfully it does not seem to have happened yet in Australia. What will likely follow from this is a more worrying loss in the perceived integrity of official statistics, a very serious outcome indeed and one the ABS will need to guard against.

As has already happened, and will continue, the demand for information, either more or new data, will increase rapidly, with the need for it to be presented more and more quickly. In addition, data by themselves will not be enough, as data from many sources will need to be integrated to throw light on the topic of interest for decision and policy making. The lesson for the ABS is not to attempt to do everything, which does not happen today, but to ensure that the market the ABS is addressing is understood and well defined, and that issues such as integrity and brand identification are cherished. In these areas the ABS is starting from a good position indeed.

One can confidently predict that the demand from governments for statistics will continue to be strong. Indeed there are signs, such as the provision of all the revenue collected from the goods and services tax to the States, that the demand from the States will increase. In addition there is a considerable thrust now for most issues to be considered on a regional basis, where the regions often cut across State boundaries. The ABS needs to be aware of these forces, and to think through how it should respond, perhaps with an emphasis on the use of analytic methods.

Increasingly, statisticians are being asked to provide measures in areas where the underlying concepts and frameworks are at best ill defined or non-existent. Before statisticians can bring facts to bear on topics such as 'the new economy', the 'knowledge-based economy', 'innovation', 'social capital', 'genuine progress', and 'sustainability', they will need to work closely with theoreticians and policy analysts to tease out the underlying policy interests and develop appropriate statistical frameworks. Indeed, even the concept of 'industry' is quite nebulous these days and a fundamental rethink of that classification is overdue.

Many people would say that the biggest changes in the future will be technology driven, and they may be right. The consensus is that the Internet will be the most significant development, and that it will play the most significant role in the development of the global village.14 As is already evident, it will change the way we work and whom we work with, how we shop, what we buy and how much we pay, and how we build personal relationships and maintain them. Most importantly, it is expected that the Internet will be the avenue for commerce for the next millennium. For statistics there have been predictions that, in the coming century, the impact of technology will be such that there will no longer be such a thing as a national economy. (That begs the question whether there is such a thing today!) The argument is put forward that there will be regional economic hubs feeding into one world economy. Even if this does not happen completely, the implications for official statistics generally are quite significant, both for assessing the validity of the current set of economic indicators and for defining the most appropriate set for the future.

Following on from this discussion, of course, are the more mundane but still important issues of collecting data electronically from respondents, both businesses and individuals, and distributing more data in this way. In addition, as more personal dealings are electronic, massive databases of information will be compiled, as indeed has already happened in Australia. Utilising these databases, such as point of sales data, will be an interesting analytic challenge and a prime objective for the ABS. Privacy will be an important issue in all these dealings.

The attitude of the community to employment is changing, and perhaps may change significantly over the next 20 years. It has often been predicted that careers within an organisation, or a few organistions, will be a thing of the past, being replaced by individuals managing their careers. This may bring with it dramatic changes to organisational design, with a large impact on organistions like the public service and perhaps businesses in Asia. The movement of employees between employers will not of course be bounded by the borders of countries; people will increasingly move from country to country in pursuit of career opportunities, as they do even today. The implication for the ABS may be great.

This discussion leads one to consider the international statistical scene, as the changes considered above, if they occur, should have massive impacts on statistical standards, and perhaps even for the collection of global statistics. It might even be possible to have international links between statistical systems; two examples worth considering might be cross checking international trade data and the flow of people in and out of countries. In my view the international scene is not in good shape at the moment and would not be capable of facing such challenges, being at its best chaotic and at its worst very ineffective. There are many players including, from the UN umbrella, the UN Statistics Division and the statistics areas of ILO, FAO, WHO, UNDP, UNFPA and UNESCO. The World Bank, the International Monetary Fund, the OECD and Eurostat also need to be counted. The problem is that there is never any joint involvement of the chief statisticians of the countries and these international agencies aimed at setting the world’s statistical agenda. Unfortunately the statisticians of the countries of the world have not played an important enough role in the management and development of international statistics policy. It is urgent that this matter be addressed, and the UN Statistical Commission might be a good starting point to do so. Globalisation in its broadest sense also makes one wonder whether there might be a need for an international statistics agency given the task of keeping the statistics for the world in order.


Conclusion

It must always be remembered that politicians the world over have a love-hate relationship with official statistics, and hence with statisticians. This is perhaps not surprising because official statistics are used by governments both for support and for illumination, and by others to judge the performance of governments and their programs. In this complex relationship it has been said that an indicator of how well a democracy is working is the degree of independence the official statistical agencies are accorded, and their performance in producing relevant and timely statistics. Currently, the ABS performs well against these criteria and we should expect the same to be the case in 100 years time.


Endnotes

1 The contents draw heavily on the special article "Australian Statisticians and the Development of Official Statistics" published in Year Book Australia 1988, and on the first chapter, Statistical Recording, in Victorian Year Book 1973.
2 Statistical Register of South Australia, 1859, Report, p. iii.
3 Victorian Year Book Containing a Digest of the Statistics of the Colony for the Year 1873 by Henry Heylyn Hayter, pp. iii-iv.
4 Colin Forster and Cameron Hazlehurst 1988, "Australian Statisticians and the Development of Official Statistics", Year Book Australia 1988, p. 33.
5 Commonwealth Bureau of Census and Statistics (CBCS) 1908, Official Year Book of the Commonwealth of Australia, 1901-1907, p. 5.
6 Op. cit., p. 6.
7 Op. cit., p. 12.
8 Unpublished and undated paper, G. H. Knibbs, The Development of the Statistical System of Australia, p. 6.
9 'JD' published in Smith’s Weekly, 9 April 1921.
10 Unpublished and undated paper, E.K. Foreman, State Governments’ Statistical Requirements - Historical Perspective.
11 For an account of the 'Integration' project see the Special Article "Australian Integrated Censuses" in Year Book Australia 1970, p. 1041.
12 For more detail on this and other initiatives in this era see the Special Article 'Two great Commonwealth Statisticians' in Year Book Australia 2000, p. xi.
13 Hermann Habermann, Two Statistical Challenges for the Future, paper presented to the 30th anniversary celebrations of the Statistical Institute for Asia and the Pacific, Tokyo, 26 August 2000.
14 Trend Letter, Vol 18, No 24, 9 December 1999.