Page tools: Print Page Print All | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
DATA QUALITY ISSUES APPENDIX
Data that has changed due to the 2017 review All data items, other than those outlined above will continue to be available. However, with the use of additional data sources, some definitional changes, and changes to the methods applied, all OAD data will change to some degree. In particular:
Table 1. Variables in OAD data that have changed (including new data sources)
3. THE 2013 REVIEW OF OAD STATISTICS In 2013, the ABS undertook a review of its Overseas Arrivals and Departures (OAD) statistics, methodology and processing systems. The primary aim of this review and system rebuild was to improve the quality of OAD data, given its importance as an input to a broad range of statistical outputs. The new system was thoroughly tested by processing over ten years of data. Imputations for missing traveller data were improved, in particular the duration of stay and the country of birth variables. The rebuild of the system resulted in a break in series, necessitating a revised time series for OAD data based on the improved methodology from July 2004 to December 2013. The break in series was from July 2004. Most of this has now been superseded due to the 2017 review which has revised the series from July 2007 onwards. 4. DATA DERIVATIONS AND IMPUTATIONS Improved methods from July 2007 During the review of the ABS' Overseas Arrivals and Departures (OAD) statistics in 2017, all derivations, logical edits and imputations were re-designed based on the best information, practices, data sources and methodology available. There were a number of derivations and imputations undertaken that improve the quality of variables with missing responses. First, the ABS utilises alternate sources where available. The sources currently available from Home Affairs include: all overseas movements data stored on the TRIPS system, monthly missing country of birth data, and monthly New Zealand passport data. An additional alternate source used for state or territory of residence is Medicare enrolment data - see 'state or territory of stay/residence' in Section 5 below for further information. For certain variables that are missing, information about an individual can be derived from these alternate sources including an individual's nearest other travel movement. This is able to be done from Home Affairs sources, by using the unique person identification number to link to the various sources and over time. For example, by accessing all movements stored on the TRIPS system, the ABS is able to make use of information from a corresponding incoming passenger card to acquire information about an individual's departure movement. Pre-reference date data includes all previously processed historical records, whereas the post-reference date data includes all TRIPS records up to and including 27 days after the end of the reference month. The variables which are available from accessing these alternate sources include: country of birth, country of citizenship, country of residence/stay, duration of stay, passenger card box type, and state of residence/stay. Second, a 'hot deck' imputation method is then used for any remaining missing responses. For 'hot deck' imputation, a record with missing responses (called the recipient), receives those of another similar record (called the donor) which has a full set of responses before the imputation process began. The recipient record keeps all of its original responses and only has the missing variables imputed, thereby keeping as much of the collected information for that record as possible. The 'hot deck' imputation method uses a set of characteristics that choose the donor and recipient records which are as similar as possible. The characteristics used within the OAD system to align a recipient with a suitable donor, vary between the different imputations. A combination of different characteristics was tested for each of the imputations to ascertain which would give the best results. The characteristics used include: age, country of birth, country of citizenship, country of residence/stay, direction of traveller, category of movement, passenger card box type, sex, state of clearance and visa group. The variables which are subject to 'hot deck' imputation are: age, country of birth, country of citizenship, country of embarkation, country of residence/stay, duration of stay, passenger card box type, reason for journey, sex and state of residence/stay, and a specific one for the country of birth of New Zealand (NZ) citizens. All missing values for these variables are fully imputed except country of residence/stay for permanent arrivals. However, from July 2017, due to the removal of the outgoing passenger card, the following are not imputed: country of disembarkation for any departures and country of stay for any Australian resident departures as both are no longer collected. For details on each variable being imputed see Section 5. 'Overview of Variables Imputed'. Missing responses Table 2 below shows the missing response rates for the month of July 2019 as an example. The rate is a percentage of the total movements for the month unless otherwise stated.
Missing responses for country of residence/stay A further breakdown in Table 3 shows the proportion of responses missing for each passenger card box type, for July 2019 as an example.
5. OVERVIEW OF VARIABLES IMPUTED Age The primary source for data on this variable is passport or visa information. An alternate source used is the incoming passenger card. Age is calculated using date of birth. For the hot deck imputation, the variables used to align the recipient with a suitable donor are: the passenger card box type and visa group. Generally, prior to using the hot deck imputation for age, the missing rate is less than 1% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Country of birth The primary source for data on this variable is passport or visa information if available. It is not available from the passenger card. The majority of imputations are for NZ citizens. There are two separate parts to the imputation for country of birth. A specific imputation is in place for the country of birth of New Zealand (NZ) citizens, as data for this variable is not directly available from the passport or visa of NZ citizens. For details see 'Specific imputation for country of birth of New Zealand citizens' in Section 6 below. The second is a hot deck imputation and is only used for non-NZ citizens. For this imputation, the variables used to align the recipient with a suitable donor are: category of movement and country of citizenship. Generally, prior to using the hot deck imputation for country of birth for non-NZ citizens, the missing rate is less than 1% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Country of Citizenship The primary source for data on this variable is passport or visa information. An alternate source used is the incoming passenger card. There are two separate parts to the hot deck imputation for country of citizenship. The first part is used if the data is missing. For this imputation the variables used to align the recipient with a suitable donor are: direction, visa group, and country of birth. The second part is only used if partial information is supplied on the passenger card. For example, if no official nationality is available and Europe was supplied as the nationality on the passenger card it would then be imputed to a country in Europe. The specific region or country grouping provided as the recipient's citizenship is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, prior to using the hot deck imputation for country of citizenship, the missing rate is less than 1% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Country of embarkation The primary source for data on this variable is the incoming passenger card. When the passenger card is missing, the alternate source is flight schedule information. Prior to July 2007, the only available source for data on this variable was the passenger card. From July 2017, the country of disembarkation for all departures is no longer available due to the removal of the outgoing passenger card. It is therefore no longer available for visitors departing or Australian residents departing. There are two separate parts to the hot deck imputation for country of embarkation. The first part is used if the data is missing. For this imputation the variables used to align the recipient with a suitable donor are: category of movement and country of residence/stay. The second part is only used if partial information is supplied on the passenger card. For example, if Europe was supplied as the country of embarkation on the passenger card it is imputed to a country in Europe. The specific region or country grouping provided as the recipient's country of embarkation is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, prior to using the hot deck imputation for country of embarkation, the missing rate averages less than 7% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Country of residence/stay The primary source for data on this variable is the incoming passenger card. From July 2017, the country of residence/stay for Australian citizens and resident departures was no longer available due to the removal of the outgoing passenger card. As outgoing passenger cards have been removed, for visitors departing the primary source used is now an individual's arrival passenger card from their previous movement. For example, the vast majority of travellers (excluding permanent migrants) have two legs to their journey, such as an arrival followed by a departure. As historical data is available, retrieving an individual's previous movement (where possible) allows for the country of residence to be collected for a visitor departing. Country of Stay for Australian citizens and residents departing is no longer available for the revised series from July 2007. An alternative data source is visa information which may be used for some travellers when available although this is limited. Prior to July 2007, the primary source for data on this variable was the original arrivals or departures passenger card (from the reference movement), and alternately visa information for some travellers. There are two separate parts to the hot deck imputation for country of residence/stay. The first part is used if the data is missing. For this imputation the variables used to align the recipient with a suitable donor are: category of movement, country of citizenship, and visa group. The second part is only used if partial information is supplied on the passenger card. For example, if Europe was supplied as the country of residence/stay on the passenger card then it is imputed to a country in Europe. The specific region or country grouping provided as the recipient's country of residence/stay is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, prior to using the hot deck imputation for country of residence/stay, the missing rate averages less than 22% of all records. Country of residence/stay is not imputed for permanent arrivals and since July 2007 for Australian citizens and residents departing. All other records are fully imputed. See Section 4. 'Data Derivations and Imputations' above. Prior to the '2013 Review of OAD Statistics' and the revision back to July 2004, the ABS imputed this data item in two stages. In the first stage, records with country of residence/stay missing were set to country of disembarkation/embarkation if a response was available. In the second stage, for remaining records where country of stay/residence was missing, values were imputed at the category of movement, reason for journey and country of citizenship level based on responses to other cards within each subgroup. For permanent arrivals, imputation was undertaken using a combination of country of embarkation and the stated responses of other permanent arrivals. Duration of stay - current Data on this variable is from two separate sources; firstly, it can be sourced directly from the traveller and secondly, it can be measured using movement dates. These methods have been applied on the revised data series from July 2007 and are outlined below.
If the reference movement is the second leg of a journey, the exact dates a traveller crosses the Australian border are both available and therefore record a traveller's actual duration of stay. This applies to visitor departures and resident returns which are second leg of journey movements. For resident departures, the reference movement is the first leg of journey and the intended duration is not available due to the removal of the outgoing passenger card. Accessing the post-reference date data (up to 27 days after the reference month) enables the measurement of the actual duration for the vast majority (usually above 80%) of resident departures. If the intended duration of stay is missing for a visitor arrival then this secondary source will also be used. Any remaining records with a missing duration of stay are imputed. The quality for actual measured duration of stay recorded at the second leg of a journey is more accurate than that based on a traveller's intended duration of stay. In the example below for resident departures in June 2017, you can see prominent spikes for intended duration of stay at 1 week, 10 days, 2 weeks, and 3 weeks which is very different to the actual measured duration. FIGURE 1 DURATION OF STAY (% of all movements), Resident Departures - June 2017 If duration is unable to be sourced from the dot points noted above then it is imputed. There are three separate hot deck imputations used for duration of stay. 1. The first imputation is used if duration of stay is missing but excludes any resident departures (see point 3 below for imputations for resident departures). For this imputation, the variables used to align the recipient with a suitable donor are: passenger card box type, country of citizenship and visa group. 2. The second imputation is only used when a visitor has put one year exactly at their intended duration of stay on the arrival passenger card. It only applies to temporary entrants. This imputation reflects historical patterns that clearly show the majority stay less than one year. The imputation first involves creating an historical data set based on information from two years earlier. It then calculates the actual recorded duration of stay for those travellers who had originally put one year exactly as their intended duration of stay. This group becomes the required donor pool. For this imputation the variables used to align the recipient with a suitable donor are: passenger card box type, country of citizenship and corresponding month. For the proportion imputed to either a long-term stay or a short-term stay for July 2017 as an example, see Table 4 below. 3. The third imputation is used when a resident departure has not had its actual duration already measured using the post-reference date data up to 27 days after the reference month (i.e. residents who had not yet returned). Similar to the second method noted above, this imputation involves creating an historical data set based on information from two years earlier. It then calculates the actual recorded duration of stay for those travellers who had originally been flagged for imputation two years earlier. This group becomes the required donor pool. For this imputation the variables used to align the recipient with a suitable donor are: passenger card box type, corresponding week of the month, country of birth and visa group. Due to this method used, duration of stay is only able to be imputed up to 1 year and 11 months. For the proportions imputed to either a long-term stay or a short-term stay for July 2017 as an example, see Table 5 below.
Classification of duration of stay by category of movement is as follows:
For a complete list of the categories of movement, see the Glossary under the Explanatory Notes tab. Duration of stay - historical Over time, there have been a number of changes to information collected on duration of stay. Initially, the intended duration of stay was only collected from information provided by all travellers on incoming and outgoing passenger cards in the intended length of stay fields. Therefore historically, the first leg and second leg of a journey both collected duration of stay based on intention. With the introduction of TRIPS by Home Affairs in July 1990, the new system made possible the calculation of the actual length of stay/absence for travellers on the second leg of their journey (i.e. departing overseas visitors and returning Australian residents). This calculation based on TRIPS data commenced in July 1998. This change resulted in an improvement in data quality for duration of stay. In particular, for the distribution of the number of passengers staying for one year exactly declining significantly for this group of travellers. The introduction of a new passenger card processing system from July 2001 provided further evidence of travellers rounding to one year exactly for their intended duration of stay in Australia or overseas. To reflect the historical movement patterns, the records with a reported duration of one year exactly were allocated to short-term or long-term. For visitors arriving in Australia, 75% of such records were allocated to short-term and 25% to long-term. For residents departing Australia, the distribution was 67% short-term and 33% long-term. With the '2013 Review of OAD Statistics', these proportional splits were able to be based on the behaviour of travellers from two years earlier - see Table 4. This method was applied to the revised data from July 2004. With the '2017 Review of OAD Statistics' and the removal of the outgoing passenger card, this method is not applicable for resident departures. It has been applied to the revised data from July 2007. There is evidence to suggest that when completing the intended duration of stay question on the incoming passenger card (Box B), some passengers are entering their arrival/departure date or their birth date rather than their intended duration of stay. From September 2003, a rule was implemented to the data processing system at Home Affairs stating that if all three elements are complete (years, months and days), then the intended duration of stay was to be coded to a non-response. Prior to July 2004, a simple assumption was put in place that set any traveller with a missing duration of stay to 10 days and therefore to a short-term movement. Missing response rates for the duration of stay are only available since November 1998. Prior to this, imputation carried out as part of processing by Home Affairs prevented reliable estimation for missing duration of stay. Passenger card box type The primary source for arrivals data on this variable is the incoming passenger card. Administrative systems at Home Affairs and the ABS are also used as an alternate source for some travellers. During the edit process some items are corrected where they conflict with other known information. For example, all travellers with a permanent arrival visa arriving for the first time would be converted to Box A (migrating permanently to Australia) or all visitors on a temporary visa would be converted to a Box B (for arrivals). Due to the removal of the outgoing passenger card in July 2017, the source for box type for departures data has changed. It has changed from a passenger declaration from the card to a derivation from the administrative data provided by Home Affairs. The definitions for Box D (Visitor departures) and Box E (Australian resident departures) have also needed to change as noted below. Box F (Permanent departures for Australian residents) is no longer available. In addition, a revised 10 year historical time series based on this definitional change was introduced from July 2007. Variables from the TRIPS system used by the ABS to derive box type for all departures include: country of citizenship, visa type, duration of stay, and box type from the corresponding incoming passenger card.
The hot deck imputation is only applied to arrival records as all departure box types are fully allocated based on the rules noted above. The variables used to align the recipient with a suitable donor are: direction of traveller, visa group and country of citizenship. From July 2004 to June 2007, sampled or non-sampled data were used instead of visa group data. In addition, the variable 'stay-intent' based on 'intention to live in Australia for next 12 months (for arrivals only)' was also used. Historically, prior to using the hot deck imputation for passenger card box type, the missing rate is less than 1% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Reason for journey The only source available for data on this variable is the incoming passenger card. Reason for journey is only available for visitor arrivals and from July 2017 for resident returns. Prior to July 2017, it was available for resident departures, however, is no longer available due to the removal of the outgoing passenger card. For the hot deck imputation the variables used to align the recipient with a suitable donor are: category of movement, sampled or non-sampled data, visa group, age, duration of stay, country of citizenship region group and mode of transport. Prior to July 2017, donors used were: category of movement, sex and age. Prior to July 2007, donors used were: sampled or non-sampled data, passenger card box type, category of movement, and age. Generally, prior to using the hot deck imputation for reason for journey, the missing rate averages less than 7% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. Sex The only sources available for data on this variable are passport or visa information. For the hot deck imputation the variables used to align the recipient with a suitable donor are: the passenger card box type and visa group. Generally, prior to using the hot deck imputation for sex, the missing rate is less than 1% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. State or territory of stay/residence The primary source for data on this variable is the incoming passenger card. From July 2017, for all departures the source used is an individual's nearest arrival movement where possible. For example, the vast majority of travellers (excluding permanent migrants) have two legs to their journey, either an arrival followed by a departure or vice versa, a departure followed by an arrival. By retrieving an individual's most recent other arrival movement (where possible) this allows for the state or territory of residence/stay to be collected for that same individual's arrival passenger card. By accessing all movements stored on the TRIPS system, the ABS is able to make use of information from a corresponding incoming passenger card to acquire information about departure movements from Australia. Pre-reference date data includes all previously processed historical records, whereas the post-reference date data includes all TRIPS records up to and including 27 days after the end of the reference month. As some departures from Australia do not have a recent corresponding incoming movement record, a small number of records do not have state of residence able to be derived from the data supplied to ABS by Home Affairs. The ABS worked closely with Home Affairs, Tourism Research Australia, the Department of Health and the Department of Human Services to identify and test alternate data sources to the OPC that could provide state of residence information. Medicare enrolments were identified as the only viable source for this information given the vast majority of Australian residents are registered with Medicare. Testing of this approach showed that linkage of movement records with Medicare enrolment records was technically feasible and allowed the continued production of high quality statistics. Medicare enrolment records are supplied to the ABS by the Department of Human Services. The information supplied to the ABS for this purpose does not include any Medicare claims or other health information. Each month, the ABS attempts to link movement records to a corresponding Medicare enrolment record to obtain state or territory of residence. This data linkage is undertaken in a dedicated facility and follows strict protocols to protect security and confidentiality. While many movement records relate to persons who are non-residents or otherwise aren't eligible for Medicare, all persons with a movement in the reference period are in scope for the linkage. Overall, approximately 57% of movement records were able to be linked to a Medicare enrolment record and state or territory of residence is able to be obtained for approximately 95% of Australian residents. Medicare enrolments information is used as the source of state of residence/stay in OAD statistics for approximately 3% of movements. If state of residence/stay is not available from any of these sources, it is imputed. For the hot deck imputation the variables used to align the recipient with a suitable donor are: the passenger card box type, state of clearance, country of citizenship and visa group. Generally, prior to using the hot deck imputation for state or territory of stay/residence, the missing rate averages less than 4% of all records. All records are fully imputed for this variable. See Section 4. 'Data Derivations and Imputations' above. 6. SPECIFIC ISSUES FOR NEW ZEALAND PASSPORT HOLDERS Allocating passenger card box type and a category of movement Under the Trans-Tasman Agreement, New Zealand (NZ) citizens are not required to have a visa to travel to Australia. As a result, on their arrival in Australia, visa documentation cannot be used to determine whether they are either a permanent migrant or a temporary visitor, or an Australian resident returning from NZ. Analysis undertaken by Home Affairs suggests that a substantial proportion of holders of NZ passports tick Box A (migrating permanently to Australia) each time they arrive in the country, causing an over count of NZ migrants entering Australia. The following edits were applied to arrival movement to correct the over counting of NZ migrants. From July 2001 to June 2002, Home Affairs coded all NZ citizen arrivals who had ticked Box A (migrating permanently to Australia) and had been to Australia previously (based on Home Affairs records) to residents returning (Box C). However, if these people were visitors previously, this recoding had the effect of incorrectly reducing the number of NZ migrants whilst at the same time incorrectly increasing the number of NZ citizen who were returning residents. This problem was overcome by coding the NZ citizens who had been changed by Home Affairs from Box A to Box C back to Box A. Since July 2002, Home Affairs has utilised a new edit system to ensure accurate measurement of permanent arrivals of NZ citizens. Where a person ticks Box A on his/her passenger card (migrating permanently to Australia), the record is verified by checking previous entries and related passenger card records, and if the person is previously recorded as a permanent migrant or resident then they will be counted as returning residents. This resulted in more accurate recording of NZ citizens who were migrating permanently to Australia and those who were residents returning. In 2007, to better measure the changes in traveller behaviour and more accurately capture and measure temporary migration, the ABS introduced improved methods for calculating net overseas migration. This is now the most appropriate source for statistics on migration into, and out of, Australia. Data is available from December quarter 2003. See Explanatory Note 71 in Migration, Australia (cat. no. 3412.0). In July 2017, due to the removal of the outgoing passenger card, the ABS made some definitional changes and introduced new methods to accommodate for the loss of this source data. Some changes specifically relate to New Zealand citizens. For example, a resident departure (previously Box E on the outgoing passenger card) now includes all Australian citizens, all permanent visa holders and any New Zealand citizen who can be identified as a resident. To identify a New Zealand citizen as a resident or a visitor, the following rules are applied. For a departure movement, if any individual (including New Zealand Citizens) has self-identified as a resident (Box C from the incoming passenger card) on their return trip (i.e. only up to 27 days after the reference month is available), then they are deemed to be a resident at the departure. In addition, the ABS is able to measure exactly how long since an individual's previous arrival. If a New Zealand citizen has been measured to be living in Australia for one year or more prior to departure, they are then identified as a resident departing. For all other New Zealand citizens departing they are deemed to be a visitor departing. A revised 10 year historical time series based on the definitional and methodological changes is available from July 2007 onwards. Specific imputation for country of birth of New Zealand citizens With the introduction of biometric passports for New Zealand (NZ) citizens in April 2005, the country of birth information was removed from the passport and replaced with a place of birth, for example Wellington, Auckland, Christchurch or Melbourne. The passport was the only source of information on the country of birth of NZ citizens travelling to, or from, Australia. For other travellers who are not NZ citizens, country of birth information can be obtained from their passport or visa information. However, visa information for most NZ citizens is not available as, under the Trans-Tasman agreement, they do not need to hold a visa prior to travel to Australia. Therefore, with the increased numbers of travellers holding NZ biometric passports, the proportion of movement records with a missing country of birth has increased substantially. For April 2005, NZ passport holders represented only 6% of the missing country of birth records, however by April 2007 this had increased to 79%. Ten years later, NZ passport holders are consistently representing 95% of these missing records. In 2013, a special imputation for country of birth of NZ citizens was introduced with a previous rebuild of the OAD system, with data revised back to July 2004. It improved country of birth statistics in OAD, and also outputs on Net Overseas Migration (NOM), and the Estimated Resident Population by country of birth. There are five steps to the process to generate country of birth when missing: 1. Prior to the hot deck imputation, if country of birth is missing for a NZ citizen the system will scan historical records of NZ citizens back to 2003 to see if there is an earlier record of the individual's country of birth. This is made possible through the use of a unique personal identifier provided to each traveller who crosses Australia's international border. This step looks for a record with a matching personal identifier and if one is found, will use the country of birth of the matched record. In 2016 approximately 78% of records with a missing country of birth are being matched with an historical record for the same individual. 2. If country of birth is still unknown after Step 1, the system will scan all previous imputations for country of birth for NZ citizens to see if there is an existing record for that individual. This ensures an individual's country of birth is only ever imputed once although they may cross Australia's international borders many times. 3. If country of birth is still unknown after Step 2, but there is a place of birth supplied on the NZ biometric passport, then a place to country of birth concordance is used. This concordance is dynamic and is updated each month from the historical time series, which is also updated monthly with additional data supplied by Home Affairs. The number of records for each place of birth, separately within each country of birth, is then determined cumulatively from the historical time series. That is, if the name of a place of birth is used in more than one country, for example - 'Wellington' can be found in Australia, Canada, NZ, South Africa, UK and the USA, then the method adds up the number of instances within each of those countries from the historical series. Where a record is missing country of birth, the imputation will consider all possible donors with a matching place of birth. It will then choose a random donor based on its probability of occurring from the concordance, and copy across the donor's corresponding country of birth. By the end of Step 3, up to 98% of NZ citizens with a missing value have been provided a country of birth. 4. If country of birth is still unknown after Step 3, but there is a place of birth supplied, then a search is done on all NZ towns and place names. If a match is found, it is assumed the country of birth of that record is New Zealand. Very few records are imputed using this step. 5. Lastly, if country of birth is still unknown for any NZ citizen after all other steps are taken, then the standard hot deck imputation is applied but only for non-New Zealand-born as it is assumed any New Zealand-born will have been picked up in the previous four steps. Currently, less than 1% of records are imputed using this step. 7. HISTORY OF PROCESSING CHANGES July 1998, Permanent Departures Prior to July 1998, the number of overseas-born (excluding NZ) permanent departures of Australian residents was overstated. In July 1998, Home Affairs introduced a box type validation edit to the processing system. The edit checked and corrected the box type according to the Visa Class/subclass. With the exception of Australian and NZ citizens, only Australian residents departing permanently (Box F) who hold permanent visas were retained in this box type. For temporary visa holders who incorrectly ticked Box F, their box type was changed to visitor or temporary entrant departing (Box D). July to December 1998, Reason for Journey Before the introduction of the redesigned passenger card in July 1998, 5% of short-term visitor arrivals, on average, were recorded as having a reason for journey of 'Other' or 'Not Stated'. This percentage rose to 14% for July, 16% in August and 29% in September 1998 as a result of processing problems. These problems were addressed by Home Affairs, with the percentage of 'Other' and 'Not Stated' dropping to 8% and 7% in October and November respectively. From January 1999, OAD statistics referencing these three months have been revised. The revised data were calculated by estimating the number of persons responding 'Other/Not Stated' using past trends for each country of citizenship and proportionally allocating any persons in excess of the estimated 'Other/Not Stated' total amongst the remaining categories. July to December 1998, State or territory of residence/stay For the months of August 1998, September 1998 and October 1998, data entry problems experienced by Home Affairs caused an overstatement of the Northern Territory as the main state of stay with a corresponding understatement for the remaining states and territories. In November 1998 these numbers returned to levels more comparable with previous years, with Home Affairs indicating that they had instigated data quality procedures to address this issue. From January 1999, OAD statistics referencing these months have been revised. The revised data were calculated by estimating the number of persons indicating the Northern Territory as their main state of residence/stay using past trends and proportionally allocating any persons in excess of these estimates amongst the remaining states and territories. With the introduction of the new processing system from July 2001, Home Affairs provided the ABS with data on all missing values for state or territory of residence/stay. From July 2001 to Jun 2004, any missing state or territory of residence/stay were imputed using category of movement and state of clearance. September 1998, Age, Country of Birth, Citizenship and Sex A problem was experienced in the processing of OAD data for movement dates between 6 September 1998 and 16 September 1998, following the introduction of changes to Home Affair's input processing system. This problem may affect around 10% of all September 1998 records used in estimation and result in incorrect details for citizenship, date of birth, sex and country of birth. September 1999, China and Hong Kong September 1999 overseas arrivals and departures data were revised for movements from, and to, China and Hong Kong in respect of three variables: country of birth, country of citizenship and country of residence/stay. Changes to 'country of birth' and 'country of citizenship' have been made from data supplied by Home Affairs. Changes to 'country of residence/stay' have been made by assuming the average proportion of country of birth to country of residence/stay for migrants from China and Hong Kong in September 1995 to September 1998. July 2004, All Data In 2013, the ABS completed a rebuild of the system which creates OAD data. At that time, all OAD data had been revised back to July 2004 based on the improved methodology. July 2007, All Data In 2017, due to the removal of the outgoing passenger card, the ABS undertook a review of its OAD statistics, methodology and processing systems. The rebuild of the new system allowed better use of a range of alternative data sources. Due to these changes, all OAD data have been revised back to July 2007 based on the new methodology and definitions - see Section 2 above. January 2013, Duration of Stay and Reason for Journey Investigations by the ABS and Home Affairs uncovered a high non-response rate for both duration of stay and reason for journey for the month of January 2013. This was mainly due to changes to the collection and processing of passenger cards, which were introduced in that month. January is the only month that was affected and the non-response rates for subsequent months were at an acceptable level. October 2014 to May 2015, Delay in release of overseas arrivals and departures statistics and missing passenger cards A number of releases of Overseas Arrivals and Departures, Australia (cat. no. 3401.0) were delayed. This was due to an attempted transition to a new provider of passenger card processing by Home Affairs. October 2014 to June 2017, Increases in Missing Passenger Cards From October 2014 to May 2015, Home Affairs trialled a transition to a new provider of passenger card processing. During this period there was a higher number of cards missing than expected. From August 2015, Home Affairs rolled out new technologies to streamline and strengthen border operations. Departure SmartGates are currently in use at all international airports in Australia for processing passengers leaving the country. Passengers using Departure SmartGates are required to place their outgoing passenger card in drop boxes located near the Departure SmartGate rather than handing them to Australian Border Force officers. Coinciding with the roll-out of Departure SmartGates, there had been an increase in outgoing passenger cards not being collected due to passengers failing to place their cards into the drop boxes. To accommodate the increases in missing passenger cards, the ABS completed an upgrade to the OAD system in March 2016 to introduce new methods and additional data sources to help maintain data quality. They included: an individual's movement information from TRIPS where no card was received (i.e. an unmatched record); retrieving and using information from an individual's most recent known future movement (movement after the reference movement) or historical movement (movement before the reference movement) where appropriate; and flight information to ascertain an individual's country of embarkation or disembarkation. This maintained the quality of most variables and reduced reliance on the use of imputations. July 2016, Norfolk Island From 1 July 2016, Norfolk island was integrated into Australia for administrative purposes. Travel between mainland Australia and Norfolk Island is no longer considered an international movement and has been excluded from the Overseas Arrivals and Departures statistics. September 2016, New Zealand Citizen long-term resident returns to Australia The ABS became aware of an increase of resident returns to Australia by New Zealand citizens with an overseas stay of 1 year or more (long-term resident returns). This increase is notable from September 2016 onwards and is not fully supported by real world explanations and has been traced to the input data ABS has received from Home Affairs. This issue has been flagged as a data quality concern and is being investigated by Home Affairs. July to November 2017, State or territory of stay/residence The ABS identified a data quality issue with state or territory of stay/residence for the period July to November 2017, with estimates for the Northern Territory being understated and a corresponding overstatement for New South Wales. The cause was identified as a coding issue with the new incoming passenger card, introduced from July 2017. Affected cards have been re-processed and revised statistics for July to November 2017 were included in the December 2017 issue of Overseas Arrivals and Departures, Australia (cat. no. 3401.0). The result of this revision is that estimates for the Northern Territory are approximately 30-40% higher in the revised data. Due to the data for these months having been re-processed, there are also minor changes to other published OAD statistics. July 2017 to February 2018, State or territory of stay/residence Investigations by the ABS and the Department of Home Affairs identified quality issues with the capture and coding of state or territory of stay/residence from incoming passenger cards from July 2017 onwards. A resolution to these issues was implemented and data re-processed. Revised OAD data from July 2017 to February 2018 was released in the March 2018 issue. For the difference between revised and previously published OAD data for Short-term visitor arrivals and Short-term resident returns by State or territory of residence/stay, see Table 6 in the Data Quality Issues Appendix in the March 2018 issue. July 2017 Main Reason for Journey In July 2017 the Department of Home Affairs discontinued the outgoing passenger card and introduced changes to the layout of the incoming passenger card in order to capture the main reason for journey for residents returning to Australia, as well as visitors arriving in Australia. As the layout of the potential responses on the card changed from this point, there are some changes to the distribution of responses to the question on main reason for journey for short-term visitor arrivals, most notably an increase in the number of persons reporting that the main reason for travel was visiting friends and relatives and a decline in the number of persons reporting the main reason was a holiday. Following an extensive investigation of this issue, ABS modelling suggests that the proportion of people reporting visiting friends and relatives is approximately 4 percentage points higher, and holiday approximately 4 percentage points lower, after the introduction of the revised card layout. Also following this investigation, improvements have been made by Home Affairs to the process of capturing and coding main reason for journey from the card, and the ABS has further reviewed and made improvements to imputation of reason for journey. Therefore, when comparing statistics on reason for journey between periods before and after July 2017, users should consider the changes noted above. March to May 2018, State or territory of stay/residence Investigations by the ABS and the Department of Home Affairs identified quality issues with the capture and coding of state or territory of stay/residence from incoming passenger cards from March 2018 onwards. A resolution to these issues was implemented and data re-processed. Revised OAD data from March to May 2018 has been released in the June 2018 issue. All time series spreadsheets and data files have been revised. For the difference between revised and previously published OAD data for Short-term visitor arrivals and Short-term resident returns by State or territory of residence/stay, see Table 6 in the Data Quality Issues Appendix in the July 2018 issue. April to July 2018, Country of Residence/stay In June 2018, the ABS identified a data quality issue with the country of residence of Bangladesh being implausibly high for short-term visitor movements. There was a corresponding decrease for Singapore and Malaysia. To fix this issue Home affairs implemented a procedural change in September 2018. For the period from April to August 2018 the ABS have identified the affected records and amended them by assigning to either Malaysia or Singapore. The amendments were based on historically observed distributions for travellers on visa subclass 601, stratified by country of embarkation and country of citizenship. Revised OAD data from April to July 2018 has been released in August 2018 issue. All time series spreadsheets and data files have been revised. For the difference between revised and previously published OAD data for Short-term visitor arrivals for Bangladesh, Singapore and Malaysia by country of residence, see Table 6 in the Data Quality Issues Appendix in the August 2018 issue. Document Selection These documents will be presented in a new window.
|