Improvements to the estimation of net overseas migration

Details about how ABS improved the net overseas migration estimation method after 1 July 2017 following the cessation of the outgoing passenger card

Released
29/05/2018

Main features

1. Introduction

In 2017, there were more than 40 million travel movements across Australia’s border. This is forecast to rise to 50 million by 2020.¹ In an initiative to create a more efficient and streamlined process for travellers departing Australia, the requirement for international travellers to complete an outgoing passenger card was removed by the Australian Government Department of Home Affairs from 1 July 2017.

As information from the outgoing passenger card was used by the Australian Bureau of Statistics (ABS) in the production of statistics on overseas arrivals and departures (OAD) and net overseas migration (NOM) (and therefore Australia’s population), the removal of the card has required the ABS to review its OAD and NOM statistics, methodologies and processing systems.

The purpose of this information paper is to provide an overview of the changes implemented to NOM estimation associated with the removal of the outgoing passenger card.

2. Background

The ABS is required by legislation to produce statistics on the population of each state on a quarterly basis. These statistics, known as the Estimated Resident Population (ERP), are critical for decisions such as the determination of seats for each state and territory in the House of Representatives and the distribution of Goods and Services Tax (GST) revenue between the states and territories.

Consistent with internationally agreed definitions, ERP is based on the concept of usual residence. For the purposes of NOM and therefore ERP, a person is regarded as a usual resident if he/she has been (or was expected to be) residing in Australia for a period of 12 months or more. As such, the ERP includes all people, regardless of nationality, citizenship or legal status, who usually live in Australia, with the exception of foreign diplomatic personnel and their families.

One component of population change used to produce ERP is NOM, as are births and deaths. Currently, NOM accounts for well over half of Australia's population growth (63% for year ending 30 June 2017).

The input data for calculating NOM is mainly sourced from administrative data provided by the Department of Home Affairs. Administrative information on persons arriving in, or departing from, Australia is collected from various sources including passport documents, visa information, and passenger cards. The NOM estimates are produced using all overseas arrivals and departures to determine how many people migrate into, and out of, Australia in each quarter. As the vast majority of movements across Australia’s border represent short term trips, only around 1% of all movements across Australia’s border contribute to NOM.

3. Previous methods for estimating NOM

Conceptually, NOM is the difference between the number of people arriving in Australia to live for 12 months or more, and the number of people leaving Australia to reside overseas for 12 months or more. The introduction in 2006 by the ABS of the '12/16 month rule' method for estimating NOM, means the 12 month duration threshold does not have to be continuous and is measured over a 16 month reference period. Records of each traveller’s overseas arrivals and departures are used to measure his/her cumulative duration in (or out of) Australia over the subsequent 16 months following an initial arrival or departure. Travellers attaining a combined duration in Australia of 12 months or more, and who are not currently counted within the population, are added to the population. Conversely, residents attaining a combined duration out of Australia of 12 months or more, and who are currently counted within the population, are subtracted from the population. The ability to accurately measure the duration of stay in or out of Australia is a key component in determining which arrivals or departures contribute to NOM, known as 'NOM Arrivals' and 'NOM Departures'.

The '12/16 month rule' can only be fully applied up to 16 months after an initial arrival or departure. However, preliminary estimates of NOM are required in order to provide more up-to-date statistics on Australia’s population. To estimate preliminary NOM, the ABS previously used a model that predominantly applied the behaviour of similar travellers from one year earlier to travellers in the quarter of interest. Travellers with similar characteristics were grouped according to specific variables and their likelihood of representing a NOM arrival or NOM departure was predicted based on the actual behaviour of similar travellers from one year earlier. Previously the variables used were: age, country of citizenship, initial category of travel, and state/territory of usual residence. The method for the estimation of preliminary NOM has now changed and is noted below in Section 6 - Improvements to Preliminary NOM Estimation.

4. Changes to input data

The removal of the outgoing passenger card and assessment of alternate data sources have resulted in changes in the data and methods used in the calculation of NOM. Of these changes, those most significant are:

• The inclusion of an additional month of movement data — this has further reduced the pool of travellers required to use the propensity model, improving the accuracy of preliminary NOM estimation.

• Changes in data sources for state/territory of residence — data previously sourced from the outgoing passenger card is now collected from alternate sources including an incoming passenger card for that traveller and information from Medicare enrolment records.

• Incorporating improvements to the personal identifier (known as PID) for movement records — updates to PID data supplied by Home Affairs each month are used to improve the quality of travel histories that are produced for each traveller. As a consistent PID is crucial for determining a traveller's actual duration in or out of the country, this has been essential in improving the quality of NOM estimation.

• Revisions to imputation methodology in the OAD system — these now flow into the variables used in the production of NOM statistics.

For detailed information on the 2017 changes in OAD data including imputation, secondary sources used, and in particular information on the state/territory of residence, see Data Quality Issues (Appendix) in the Methodology in Overseas Arrivals and Departures, Australia (cat. no. 3401.0).

5. Improvements to final NOM estimation

The NOM methodology, based on the '12/16 month rule', remains substantively unaltered. The changes to final NOM estimates in the new system are primarily attributable to changes to input data used for NOM such as updates to the quality of PIDs, the use of alternate sources, and changes to imputation.

Improvement in the quality of PID data is a significant change. While in the vast majority of cases a traveller has all his/her movements associated with a single personal identifier, inevitably in some cases a traveller is inadvertently assigned more than one PID. Home Affairs now supplies ABS with a monthly file listing instances where these situations have been identified and rectified. This enables the ABS to more accurately construct person-level data when using the movement records from the 16 months following an overseas movement. A more accurate overall traveller count is achieved through reduced duplication. As a consequence, the number of both NOM Arrivals and NOM Departures is slightly lower than under the former NOM system.

As part of the 2017 review of OAD, the imputation methodology for missing values was redesigned. The values from these improved imputations are now carried forward into the data used in NOM processing, with the exception of instances where state/territory of residence required imputation.

Changes to the source and imputation of state/territory of usual residence in NOM

Derivations for the state/territory of residence are conducted within the NOM system due to the availability of additional data in the time series (e.g. up to 16 months of post-reference data), and data sources used for this variable.

i. The ABS has developed a new business rule to determine the state/territory of residence for travellers determined to be NOM Arrivals (i.e. persons who have migrated to Australia). In these cases, the state/territory from the traveller's latest known movement in the 16 months following the reference movement is applied to his/her NOM Arrival record for the reference quarter. In essence, the state/territory reported from a traveller’s subsequent movement is considered to be a more accurate indication of the state/territory that a new migrant has settled in than his/her initial address in Australia following his/her arrival.

ii. Where possible, the primary source for data on the state/territory of residence for an individual is the incoming passenger card associated with the reference movement. Prior to July 2017, for all departures, state/territory of residence was sourced from the outgoing passenger card.

iii. However, as outgoing passenger cards are not available from July 2017, for all subsequent departures state/territory is sourced where possible, from an incoming movement by that individual. For example, the vast majority of travellers have two legs to their journey, either an arrival followed by a departure or vice versa. Where a recent arrival movement exists for that same individual, the state/territory of residence reported for that incoming movement is assigned to the outgoing movement. The same method is applied in those cases where state/territory is missing from an arrival record. This nearest movement search is first undertaken in the OAD processing system and then again in the NOM processing system.

iv. While the existing incoming passenger card information is able to provide state/territory of residence for most movements, a small number of records do not have state/territory able to be derived from another movement by that individual. In order to obtain state/territory of residence for those cases where it cannot be obtained from a movement record, since July 2017, the ABS attempts to link traveller movement records to a corresponding Medicare enrolment record supplied to the ABS by the Department of Human Services. This mainly captures Australian residents who are recorded in Medicare.

v. If state/territory of residence is still missing after the steps above have been applied, it is imputed. The method used is known as a ‘hot deck’ imputation, where each missing value is replaced with an observed response from a similar unit or ‘donor’. The variables used to align the recipient with a suitable donor are: broad category of travel, state of clearance, country of citizenship and visa group.

The data sources and imputations for state/territory of residence are applied using a priority order of preference as defined in Table 1 which correlates to the 5 points above.

Table 1. Priority order of preference for selecting state/territory or usual residence
PriorityData sourceSelectionApplied to
1Passenger cardLast known movement in 16 month windowNOM Arrivals
2Passenger cardPassenger card associated with the reference monthNOM Arrivals (& NOM Departures before July 2017)
3Passenger cardNearest movement searchNOM Arrivals and NOM Departures
4Medicare enrolmentsMatch to reference periodNOM Arrivals and NOM Departures
5Imputation - 'hot deck'Any traveller still with missing state/territoryNOM Arrivals and NOM Departures

Improved final estimates

• Testing of the new NOM systems (using over 10 years of data) has provided improved quality of final NOM estimation through improved accuracy in the calculation of time in or out of the country through the use of improved PID data.
• New methods, new data sources and additional data available in the time series have also improved the ability to estimate NOM by state/territory of residence more accurately.
• Data from the new processing system is likely to reduce the intercensal difference for the 2011-16 period.

6. Improvements to preliminary NOM estimation

The ABS is now able to source an additional month of movements data from Home Affairs for use in preliminary NOM estimation. Using this additional data source allows a total of four months of movement data to be available after the preliminary reference quarter. This means for many individuals that their ERP status can be resolved. This additional month has been utilised by the ABS to make two significant improvements in preliminary NOM methodology:

1. Reducing the pool of travellers using the propensity model.
2. Allowing a one year ago propensity model to be used for all remaining travellers.

These two improvements are further enhanced by use of the new data sources and imputation improvements outlined earlier for final NOM estimation. To estimate preliminary NOM, the ABS now uses an improved propensity model that estimates a traveller's propensity to contribute to NOM using the observed behaviour of similar travellers from one year earlier. Like the previous model, travellers with similar characteristics are grouped according to specific variables. However, the variables now used are: age, country of citizenship, direction of first and last movement in the reference quarter, initial ERP status, time spent out of Australia, and visa group. To create the preliminary NOM estimates, the migration propensities determined for the donor groupings from one year earlier are applied to similarly grouped travellers from the reference quarter being estimated.

Reducing the pool of travellers using the propensity model

The majority of travellers in a given reference quarter take a trip of short duration which will not change their ERP status. Most of these short-term travellers can now have this determination made based on the movements data available at the time of preliminary NOM estimation. For example, an Australian resident counted in the ERP at the beginning of a quarter, may commence a holiday during that quarter and then return to Australia two weeks later. Once four months since his/her return has passed without a further departure, he/she can be determined to still be in Australia’s population at the end of the reference quarter. This is because, as counted from the date of their original departure, it will no longer be possible for him/her to be out of Australia for more than 12 months out of 16.

The increase of available post-reference date data to four months has reduced the proportion of travellers requiring preliminary NOM estimation by the propensity model to less than 30%. This has significantly increased the predictive accuracy of the model.

Changing to one year ago propensity model for all travellers

The previous preliminary NOM propensity model only had 15 months of movement data available to determine the NOM behaviour of travellers from one year earlier. As the full 16 months required were not available to finalise the NOM status for all travellers, durations could not be directly calculated for all travellers in the quarter from one year earlier and a combination of 'one year ago' and 'two year ago' propensities was applied to travellers in the reference quarter.

Availability of an additional month of movements data now means that 16 months of data are available. This enables the calculation of duration for all travellers from the corresponding quarter one year earlier. For example, when estimating preliminary NOM for September quarter 2017, the full quarter of final NOM results for September quarter 2016 is now available for the propensity model to use. To estimate preliminary NOM, the new methodology now only uses a 'one year ago' propensity model and, as such, improves the currency of the propensities being used.

Improved preliminary estimates

• Testing of the new propensity model (using over 10 years of data) shows, that on average, preliminary NOM estimates are 50% closer to final NOM estimates.
• As NOM is currently the main component of population growth, this then also greatly improves the quality of preliminary estimates of the resident population (ERP) of Australia.
• The new methodology also helps to better estimate the turning points for preliminary NOM than in the past, as shown in Figure 1 below.

7. Changes to revision timetable for NOM & implementation dates

Estimates of both the resident population (ERP) and NOM for Australia and each of the states and territories are published quarterly in Australian Demographic Statistics (cat. no. 3101.0). The improvements outlined for preliminary NOM in this paper will be introduced from the September 2017 issue. Changes made to final NOM and a subsequent technical note will be released in the December 2017 issue on 21 June 2018.

The quarterly variability always experienced in Australia's population growth is predominantly driven by trends and seasonality in NOM. To help reduce the impact of revisions to ERP, the ABS is now able to release Final NOM one quarter earlier than was previously possible. Previously there was a lag of five quarters between the release of preliminary NOM and the release of final NOM estimates. With the availability and use of an additional month of movements data, the ABS can reduce this lag to only four quarters.

The first quarterly revision cycle for publishing final NOM will be for December 2016 in the December 2017 issue of Australian Demographic Statistics (cat. no. 3101.0) scheduled for release on 21 June 2018. This release will also include the final NOM estimate for September 2016.

Historical NOM data from September quarter 2011 onwards has been produced based on the new methods. The ABS intends to apply the series using this improved method to the 2011-2016 intercensal period as part of the final rebasing of population estimates to be released in Australian Demographic Statistics, December 2017 (cat. no. 3101.0) on 21 June 2018.

8. Further information

For information on previous improvements and changes to the estimation of net overseas migration see the following papers:

Information Paper: Further Improvements to Net Overseas Migration Estimation (cat. no. 3412.0.55.002)

Information Paper: Improving Net Overseas Migration Estimation (cat. no. 3412.0.55.001)

Information Paper: Improved Methods for Estimating Net Overseas Migration (cat. no. 3107.0.55.003)

Acknowledgments

The ABS data referred to throughout this paper is sourced from data provided by the Department of Home Affairs and the Department of Human Services and from information provided by travellers. Their continued cooperation and support is highly valued and appreciated; without it, the wide range of statistics available on overseas arrivals and departures, net overseas migration and population published by the ABS would not be available. All data received by the ABS is treated in strict confidence, as required by the Census and Statistics Act 1905.

¹ Dutton, P (Minister for Home Affairs & Minister for Immigration and Border Protection) Media release, 25 June 2017, viewed 15 March 2018 http://minister.homeaffairs.gov.au/peterdutton/Pages/removal-of-the-outgoing-passenger-card-jun17.aspx

Glossary

Show all

12/12 month rule

A method for measuring an overseas traveller's duration of stay or absence in which the 12 month usual residence criterion in population estimates is measured across a 12 month period. Under a 12/12 month rule, overseas travellers must be resident in Australia for a continuous 12 month period or more to be included in the estimated resident population. Similarly, Australian residents travelling overseas must be absent from Australia for a continuous 12 month period or more to be removed from the estimated resident population.

12/16 month rule

A method for measuring an overseas traveller's duration of stay or absence which takes an approach to measure usual residence that does not have to be continuous, as opposed to the continuous approach used under a '12/12 month rule'. Under a '12/16 month rule', incoming overseas travellers (who are not currently counted in the population) must be resident in Australia for a total period of 12 months or more, during the 16 month follow-up period to then be included in the estimated resident population. Similarly, those travellers departing Australia (who are currently counted in the population) must be absent from Australia for a total of 12 months or more during the 16 month follow-up period to then be subtracted from the estimated resident population.

The 12/16 month rule therefore takes account of those persons who may have left Australia briefly and returned, while still being resident for 12 months out of 16. Similarly, it takes account of Australians who live most of the time overseas but periodically return to Australia for short periods.

Australian resident

For estimated resident population statistics, the Census year population estimates classify a person as an Australian resident if the person has (in the most recent Census) reported a usual address in Australia where the person has lived or intends to live for six months or more in the Census year. The post-censal estimates, while based on the Census data, are updated with international migration data that have a criterion of one year or more of intended stay in or departure from Australia.

Census

The complete enumeration of a population at a point in time with respect to well-defined characteristics (e.g. Persons, Manufacturing, etc.). When the word is capitalised, "Census" usually refers to the national Census of Population and Housing.

Country of citizenship

Country of citizenship is the nationality of a person. For Overseas Arrivals and Departures data it is usually taken from a traveller's passport or visa information and in some cases from the passenger card.

Emigration

The process of leaving one country to take up permanent or semi-permanent residence in another.

Estimated resident population (ERP)

The official measure of the population of Australia is based on the concept of usual residence. It refers to all people, regardless of nationality, citizenship or legal status, who usually live in Australia, with the exception of foreign diplomatic personnel and their families. It includes usual residents who are overseas for less than 12 months over a 16 month period. It excludes overseas visitors who are in Australia for less than 12 months over a 16 month period.

Estimates of the Australian resident population are generated on a quarterly basis by adding natural increase (the excess of births over deaths) and net overseas migration (NOM) occurring during the period to the population at the beginning of each period. This is known as the cohort component method, and can be represented by the following equation:

$$\operatorname {P_{t+1} = Pt + B \ - \ D + NOM}$$, where:

$$\operatorname {P_{t}}$$ = the estimated resident population at time point $$\operatorname {t}$$

$$\operatorname {P_{t+1} }$$ = the estimated resident population at time point $$\operatorname {t+1}$$

$$\operatorname {B}$$ = the number of births occurring between $$\operatorname {t}$$ and $$\operatorname {t+1}$$

$$\operatorname {D}$$ = the number of deaths occurring between $$\operatorname {t}$$ and $$\operatorname {t+1}$$

$$\operatorname {NOM}$$ = net overseas migration occurring between $$\operatorname {t}$$ and $$\operatorname {t+1}$$.

For state and territory population estimates, an additional term is added to the equation representing net interstate migration ($$\operatorname {NIM}$$) occurring between $$\operatorname {t}$$ and $$\operatorname {t+1}$$, represented by the following equation:

$$\operatorname {P_{t+1} = Pt + B \ - \ D + NOM + NIM}$$.

Final intercensal difference (previously referenced as intercensal discrepancy)

Final intercensal difference (previously referenced as intercensal discrepancy) is the final difference between two estimates at 30 June of a Census year population: the first based on the latest Census, and the second arrived at by updating the 30 June estimate of the previous Census year with intercensal components of population change. It is caused by differences in the start and/or finish population estimates and/or in estimates of births, deaths or migration in the intervening period which cannot be attributed to a particular source. For further information see Population Estimates: Concepts, Sources and Methods (cat. no. 3228.0.55.001).

Immigration

The process of entering one country from another to take up permanent or semi-permanent residence.

Initial category of travel

Overseas Arrivals and Departures (OAD) data are classified according to permanent status or length of stay (in Australia or overseas), as recorded by travellers on passenger cards, or derived with reference to previous border crossings. Although the initial category of travel has not been used in the calculation of NOM estimates since June quarter 2011, it is still available as a variable to show how the individual was identified in OAD data on his/her arrival to, or departure from, Australia.

There are three main categories of movement and 10 sub-categories:

• permanent movement;
• permanent arrival (PA)
• permanent departure (PD) - only available prior to September quarter 2011.

• long-term movement - has a duration of stay (or absence) of one year or more; and
• long-term resident return (LTRR)
• long-term visitor arrival (LTVA)
• long-term resident departure (LTRD)
• long-term visitor departure (LTVD).

• short-term movement - has a duration of stay (or absence) of less than one year;
• short-term resident return (STRR)
• short-term visitor arrival (STVA)
• short-term resident departure (STRD)
• short-term visitor departure (STVD).

Intended length of stay

On arrival in Australia, all overseas visitors are asked to state their 'Intended length of stay in Australia'.

Migrant - international

An international migrant is defined as "any person who changes his or her country of usual residence" (United Nations 1998). The country of usual residence is the country in which a person lives, that is to say, the country in which he or she has a place to live where he or she normally spends the daily period of rest. A long-term international migrant is a person who moves to a country other than that of his or her usual residence for a period of at least a year (12 months), so that the country of destination effectively becomes his or her new country of usual residence.

In Australia, for the purposes of estimating net overseas migration, and thereby the official population counts, a person is regarded as a usual resident if they have been (or expected to be) residing in Australia for a period of 12 months or more over a 16 month period.

Migration

The movement of people across a specified boundary for the purpose of establishing a new or semi-permanent residence. Migration can be international (migration between countries) and internal (migration within a country).

Net overseas migration (NOM)

Net overseas migration is the net gain or loss of population through immigration to Australia and emigration from Australia. Under the current method for estimating final net overseas migration this term is based on a traveller's actual duration of stay or absence using the '12/16 month rule'. Preliminary NOM estimates are modelled on patterns of traveller behaviours observed in final NOM estimates for the same period one year earlier. NOM is:

• based on an international traveller's duration of stay being in or out of Australia for 12 months or more over a 16 month period;
• the difference between:
• the number of incoming international travellers who stay in Australia for 12 months or more over a 16 month period, who are not currently counted within the population, and are then added to the population (NOM arrivals); and
• the number of outgoing international travellers (Australian residents and long-term visitors to Australia) who leave Australia for 12 months or more over a 16 month period, who are currently counted within the population, and are then subtracted from the population (NOM departures).

NOM arrivals

NOM arrivals are all overseas arrivals that contribute to net overseas migration (NOM). It is the number of incoming international travellers who stay in Australia for 12 months or more over a 16 month period, who are not currently counted within the population, and are then added to the population.

Under the current method for estimating final net overseas migration this term is based on a traveller's actual duration of stay using the '12/16 month rule'.

NOM departures

NOM departures are all overseas departures that contribute to net overseas migration (NOM). It is the number of outgoing international travellers (Australian residents and long term visitors to Australia) who leave Australia for 12 months or more over a 16 month period, who are currently counted within the population, and are then subtracted from the population.

Under the current method for estimating final net overseas migration this term is based on a traveller's actual duration of absence using the '12/16 month rule'.

Overseas arrivals and departures (OAD) refer to the recorded arrival or departure of persons through Australian air or sea ports (excluding operational air and ships' crew). Statistics on OAD relate to the number of movements of travellers rather than the number of travellers (i.e. the multiple movements of individual persons during a given reference period are all counted).

Overseas migration

See net overseas migration (NOM).

Passenger card

Passenger cards are completed by nearly all passengers arriving in Australia. Information including: country of previous residence, intended length of stay, main reason for journey, and state or territory of intended stay/residence is collected. An example of the current Australian passenger card is provided with the monthly Overseas Arrivals and Departures, Australia (cat. no. 3401.0) publication in the Methodology.

Population growth

For Australia, population growth is the sum of natural increase and net overseas migration. For states and territories, population growth also includes net interstate migration. After the Census, intercensal population growth also includes an allowance for intercensal difference.

Resident

See Australian resident.

State or territory of usual residence

State or territory of usual residence refers to the state or territory of usual residence of:

• the population (estimated resident population);
• the mother (birth collection); and
• the deceased (death collection).

In the case of overseas movements, state or territory of usual residence refers to the state or territory regarded by the traveller as the one in which he/she lives or has lived. State or territory of intended residence is derived from the intended address given by settlers, and by Australian residents returning after a journey abroad. Particularly in the case of the former, this information does not necessarily relate to the state or territory in which the traveller will eventually establish a permanent residence.

Usual residence

Usual residence within Australia refers to that address at which the person has lived or intends to live for a total of six months or more in a given reference year.

Visa

Permission or authority granted by the Australian government to foreign nationals to travel to, enter and/or remain in Australia for a period of time or indefinitely.

Abbreviations

Show all

 ABS Australian Bureau of Statistics cat. no. catalogue number ERP estimated resident population GST Goods and Services Tax Home Affairs Australian Government Department of Home Affairs NOM net overseas migration OAD overseas arrivals and departures PID personal identifier

Previous catalogue number

This release previously used catalogue number 3412.0.55.004.