Data opportunities to help guide welfare reform

David W Kalisch, Australian Statistician

International Conference on Welfare Reform

ANU, 17 September 2015


Good afternoon. Thank you to the organisers for the opportunity to participate in this conference

I would like to start by acknowledging the Ngunnawal people, the traditional owners of the land we are meeting on today. I wish to pay my respects to their elders, both past and present and also acknowledge any other Aboriginal and Torres Strait leaders with us today.

While I am the Australian Statistician, head of the ABS, I will also speak from the context of long-term interest and experience in employment policy and welfare reform since the 1980s

Welfare reform context

My remarks are drawn from a broad view of welfare and welfare reform.

I am not just referring to data or research in the context of our social security system or employment assistance, in their narrow forms.

In Australia, with shared and separate responsibilities for the whole gamut of social protection and employment assistance across the Commonwealth government, state and territory governments and the non-government sector, the research and data landscape is certainly more complex.

But this also better reflects the complexity of people’s lives with many dimensions and experiences contributing to the outcomes we observe and how those outcomes change over time

Importantly, in the Australian context, we need to recognise the linkages and inter-dependencies between the impacts from different levels of government and the community sector, particularly around funding and contractual relationships. Policies and services delivered across these sectors do interact to either deliver a coherent response to the complexities of the lives of individuals and households, or alternatively we observe gaps and inconsistencies across this policy and service landscape.

This complexity of our welfare system and of welfare reform are the reasons why I believe information, in the form of data and research, are so important in this area of policy and service delivery – to assist with insight and understanding.

Nature of the data and research we require

In my remarks, I will focus on a number of levels of information and information for a number of distinct purposes.

I would suggest there are three quite distinct purposes for data and research insights in this area of Welfare Reform:

1. In the first instance, we should properly understand the environment in which welfare reform is being considered, covering past experience, current dimensions and likely future developments.

In the welfare reform context, this includes the very important dimension of economic and employment developments, but should also be broader to canvass household and family structures, income and wealth, risk factors contributing to social disadvantage and major protective factors.

Welfare reform is introduced into the real economy and society, not a modelling vacuum, and this environment can change. When there is a significant shift in the environment, there should at least be some reconsideration of the best policy approach.

For example, I would suggest that the key dimensions of the policy approach could be quite different in an economy that is achieving above trend growth or alternatively below trend growth. Some policy aspects may be the same but other aspects should be tailored to the overall economic and employment environment, and where the new opportunities are emerging.

This understanding of the current and prospective environment will also help establish the burning platform for reform.

2. Secondly, what do we know about the outcomes from prevailing policy and service delivery interventions?

If we are to design welfare reform, a good place to start is what we know about the effectiveness of existing strategies. What we know about past programs, or approaches that have been tried overseas?

I would suggest that we should take a broad, expansive, inquisitive approach.

In the Australian context, a good place to start is to understand what we do know and what we do not know about the outcomes and impact from our social services.

As a country, we spend a lot of taxpayers dollars on social security, employment assistance and other social services. We should have a good idea of what the outcomes are from these billions of dollars of spending.

This goes beyond just a simple understanding of who is getting the assistance and for how long.

I would suggest that some of the key research questions we should seek to answer include the following:

    • Are there any substantive economic impacts from these programs – are they net positive or net negative once you take account of the economic costs of funding public programs?
    • What are the long-term outcomes from these programs for the individual and their household/family?
    • Do these programs mainly have a distributional impact, which by itself may be enough to justify the interventions?
    • Are there ways interventions can be designed to encourage greater self-reliance and use economic incentives more than compliance or regulatory arrangements? Do these programs work with or act against other market adjustment forces?
    • What are the outcomes for local communities from these interventions, across a wide range of metrics? Go beyond the individual and the family to the local region.
My view is that, in this area of welfare reform, Australia does not have a strong history of comprehensive economic evaluations to draw upon. We have many of the relevant data sources, and have the opportunity to draw this information together so it is more useful and provides clearer insights around the impacts from interventions.

There is certainly some evidence available also from overseas that we can also draw upon.

For example, the US welfare reform experiments of the 1980s provide a good case study of the US using natural experiments, allowing variation in policy and services across states and measuring employment, education and income outcomes for the household over the following five years.

Still not perfect by any means, but measuring outcomes over a long period of time is critical to an understanding of the sustainability or otherwise of the program intervention. In the US context, income provides a guide to the quality of the job, a dimension that is also relevant in Australia with our means testing of social security and progressive income tax system.

Overseas evidence can be of some use, but we also need to appreciate how much of the findings are fit for purpose within the Australian context. They can be a guide to possible outcomes here, but we also need to consider how we need to exercise our judgments around the applicability in Australia.

In the case of extensive government programs, an economic approach that also takes account of the costs of raising additional revenue is a legitimate component of the exercise. In the case of employment programs, we should also consider displacement and substitution effects across the broader labour market before we call ‘success’ too quickly.

Inevitably, however, there are some things we do not know, and governments cannot wait for ‘perfect’ information before they do anything.

But at the end of the day, there is considerable upside to planning and funding more comprehensive program evaluations, so at least in the future, we will have better evidence base to design future programs and policies.

This requires greater foresight by governments to properly fund comprehensive evaluations, with information likely to be available for decision makers in many years’ time. Every subsequent government should benefit from foresight shown by previous governments to fund robust evaluations.

3. Adding to the other two dimensions I have noted of

    • Understanding the environment
    • Understanding the impact of current interventions
is the third dimension of anticipating/estimating the likely impact of proposed future policy and service strategies.

Again from the backdrop of imperfect or inadequate evaluation information, there should be an assessment of the likely impacts of policy or program changes.

At least in Government processes, there is an expectation that the policy process will require a fully developed case for the proposed intervention. This may involve both quantitative and qualitative assessments of the anticipated impacts flowing from the change, both of which can be legitimate in the context of imperfect information and in some instances novel policy design.

We may have some information to draw upon, but related to my comments about the deficiencies of past evaluations, this provides a starting point, and policy designers and proposers in many instances would also need to canvass and estimate a range of possible effects that a policy change might have.

In the context of welfare reform, the assessment of likely impacts should be broad. This does not presume the impacts will be strong or noticeable across all dimensions of employment, household income, household wellbeing and social cohesion.

In fact, past experience suggests that program impacts are usually quite modest. However, what I am suggesting is that a critical part of policy design and advocacy should be effective use of available research and data to estimate the likely impacts expected from a policy change.

This then also provides a basis for assessing success or otherwise of the policy if it is agreed and implemented.

This appears to provide a good reason for the strategy to under-promise and over-deliver in this (and any) policy environment

Broad data and information opportunities

So, if these are the dimensions that will assist us in our quest for better welfare reform, how well placed is our existing information base?

A good place to start is to understand the information we do have, and how it could be used.

I will share my conclusion with you and then explain how I got to that point.

My conclusion is that we have limited knowledge around these matters, but that the data resources and capabilities that we have at our disposal should place us in a much better position than we find ourselves currently.

There is considerable upside, and I would suggest that we can get into a much better space with focus on some early deliverables, goodwill from respective parties and not much in the way of additional resourcing.

So what do we have in the way of potential data resources, at least within government, that is pertinent to welfare reform:

    • Comprehensive information on the social security population, their type of benefits, level of payments, reported income, dependents, etc. We have the ability to construct longitudinal histories from this administrative data.
    • Information from our Job Services providers of those who have received assistance and the type of assistance they received, with some information on short term employment or education outcomes.
    • Information on those who receive assistance from a range of social services – child care, housing and homelessness services, child protection services, disability services, etc
    • Participation in education and training
    • Access to health services, including mental health services
The data held by governments as a by-product of the administration of these extensive programs and services is a valuable resource. However, this information is managed and stored within its bureaucratic silos and of limited use to governments and the community in this form.

An Australian Integrated Data Resource?

This government administrative data becomes a more valuable resource if it is combined with other information to provide more insights.

For example, administrative data can be combined, such as information on social security recipients with child care and employment services, or Commonwealth MBS/PBS information with state government hospital services to design better policy or service delivery responses.

Administrative data can also be combined with the five yearly ABS census information to provide a more comprehensive picture of changes in Australian households. Other ABS surveys of households and businesses provide further opportunities for constructing linked data sets that deliver additional insights.

ABS has taken some early steps in data integration.

    • We have a robust Commonwealth government framework for data integration, with the ABS accredited as an Integrating Authority in April 2012,
    • We have a strong legislative basis to ensure the protection of sensitive personal information, and have a strong level of community trust,
    • We have the technological and methodological expertise to undertake such projects, and
    • We have invested in maximising the use of our own vast data holdings, for example through the integration of Census data with itself over time to create a longitudinal view or with other data to provide a richer cross sectional view.
This has already led to new insights from projects on migrant settlement outcomes, the usage of mental health services and medication as well as labour market participation and transitions into and out of employment through a longitudinal labour force data set. We have also produced an experimental linked employer-employee data set, bringing together personal and business tax data. From a firm level perspective the ABS has already created a rich linked data resource of small and medium businesses allowing analysts to monitor and evaluate the impacts of policies and industry trends on Australian businesses at the micro level

Of course, in addition to good data, we also need good social researchers that can give the data meaning in the context of institutional systems and policy settings. The recent investment in an ARC Centre of Excellence for Children and Families over the LifeCourse provides an important opportunity in this respect. The ABS is working with the LifeCourse Centre partnership network to deliver enabling data and technology that will underpin this area of research.

New Zealand is already a few years’ ahead of us, with their Integrated Data Infrastructure (IDI), hosted by Statistics NZ. They established a prototype of a linked longitudinal dataset by the end of 2011, which included economic, education, migration and business data, and it was extended to include justice and health and safety data by August 2013.

Information from 11 New Zealand Government departments is combined with Statistics NZ survey information to provide a powerful resource, used by government and the research community. This has already prompted a large and wide ranging number of research projects, available on the Statistics NZ website (, and is now central to NZ Government decision making.

We have the basis for an Integrated Data Resource in Australia, and the ABS is progressing some further data integration projects, with key Commonwealth agencies in 2015-16 to include social security information and Commonwealth MBS/PBS health data. The ABS is prioritising this activity, as we resource a number of key enduring data linkage projects. It is intended that these projects will deliver publically-available data resources for research that is relevant to policy development and service design.

To get the usefulness of the prevailing NZ IDI, in the Australian context we should also include key state government services, such as hospital services, other community services and schools.

I would hope that, in several years’ time, we would have an Integrated Data Resource similar in usefulness to what the NZ government achieved in August 2013 . This is a realistic goal. Technology, expertise and confidentiality are not the issues or the constraints. It can take some time and resources for government agencies to provide better access to their data, even to an organisation such as the ABS with all the data protections and community support you would require.

This would provide a data resource that would not just be useful for the design and implementation of welfare reform, in its broader form, but for many other prevailing economic and social policy questions likely to confront Australia over coming decades.