|Page tools: Print Page Print All|
APPENDIX 2 DATA QUALITY ISSUES
NEW ZEALAND CITIZENS
Under the Trans-Tasman Agreement, New Zealand (NZ) citizens are not required to have a visa to travel to Australia. As a result, on arrival in Australia their visa documentation cannot be used to determine whether they are either a permanent migrant or a temporary visitor, or an Australian resident returning from NZ. DIAC believes that a substantial proportion of holders of NZ passports tick Box A (migrating to Australia) each time they arrive in the country, causing an overcount of NZ migrants entering Australia.
The following edits were applied to correct the overcounting of NZ migrants:
July 2001 to June 2002
With the introduction of the new processing system from July 2001, DIAC coded all NZ citizen arrivals who had ticked Box A and had been to Australia previously (based on immigration records) to resident returning (Box C). If these people were visitors previously, this recoding had the effect of incorrectly reducing the number of NZ migrants whilst at the same time incorrectly increasing the number of NZ citizen returning residents. This problem was overcome by moving the NZ citizens who had been changed by DIAC from Box A to Box C back to Box A.
July 2002 onwards
From July 2002, DIAC has utilised a new edit system to ensure accurate measurement of Permanent Arrivals of NZ citizens statistics. Where a person ticks Box A on his/her passenger card (first arrival as a migrant), the record is verified by checking previous entries and related passenger card records, and if the person is previously recorded as a migrant or resident then they will be counted as returning residents. This resulted in more accurate recording of NZ citizens who are migrating to Australia as against those who are residents returning.
INTENDED LENGTH OF STAY/TIME AWAY FROM AUSTRALIA
Non-response rates are available for these data items from November 1998. For data prior to November 1998, imputation carried out as part of processing by DIAC has prevented reliable estimation of non-response rates for these two data items.
MAIN REASON FOR JOURNEY
Before the introduction of the redesigned passenger card in July 1998, 5% of short-term visitor arrivals, on average, were recorded as having a reason for journey of 'Other' or 'Not Stated'. This percentage rose to 14% for July, 16% in August and 29% in September 1998 as a result of processing problems. These problems have now been addressed by DIAC, with the percentage of 'Other' and 'Not Stated' dropping in October 1998 to 8% and 7% in November 1998.
From the January 1999 issue of this publication, published figures (table 6 in this issue) referencing these three months were revised. The revised data were calculated by estimating the number of persons responding 'Other/Not Stated' using past trends for each country of citizenship and proportionally allocating any persons in excess of the estimated 'Other/Not Stated' total amongst the remaining categories. 'Not Stated' rates are now separately available from February 1999 onwards.
STATE WHERE SPENT MOST TIME
For the months of August 1998, September 1998 and October 1998, data entry problems experienced by DIAC caused an overstatement of the Northern Territory as the main state of stay with a corresponding understatement for the remaining states and territories. In November 1998 these numbers returned to levels more comparable with previous years, with DIAC indicating that they had instigated data quality procedures to address this issue.
From the January 1999 issue of this publication, published figures (table 11 in this issue) referencing these months were revised. The revised data were calculated by estimating the number of persons indicating the Northern Territory as their main state of stay using past trends and proportionally allocating any persons in excess of these estimates amongst the remaining states and territories.
With the introduction of the new processing system from July 2001, DIAC has provided the ABS with data on all missing values of state of stay and state of usual residence. These missing values are now imputed.
Data were imputed for non-response for state of stay/residence. For state of stay, non-responses were imputed at the category of traveller and state of clearance level. Non-response rates for state of stay are presented in the table below:
Non-responses for country of stay and country of usual residence were imputed in two stages. In the first stage, records with country of stay/residence missing were set to country of disembarkation/embarkation if a response was available. In the second stage, for remaining records where country of stay/residence was missing, values were imputed at the category of traveller, reason for journey and country of citizenship level based on responses to other cards within each subgroup. Accordingly, the level of records with data for country of stay/residence not stated has been minimised.
Change in approach to non-response state of stay for long-term visitor departures
A procedure has been applied before prorating of a non-response to state of stay for long-term visitor departures. If a correction to the box marked by a passenger is made (e.g. a visitor marks a resident box), the state of stay recorded in the incorrect box is applied.
Country of stay
Table A3 below presents the percentage of records with country of stay/residence missing as supplied by DIAC and prior to imputation.
Table A4 below shows the non-response rates for country of stay/residence following the application of the first stage of imputation.
Country of previous residence
The proportion of permanent arrivals where country of previous residence was not stated increased markedly over the two years to August 2006. Prior to August 2006 ABS imputed this data item for these movements using country of embarkation information as well as stated responses of other permanent arrivals. As a result of this increase and the continuing high level of not stated responses (see table A4), records of permanent arrivals whose country of previous residence was not stated have not been imputed for August 2006 data onwards. For all other arrivals, imputation based on country of embarkation has continued.
Country of birth for New Zealand passport holders
With the introduction of biometric passports for NZ passports, from 21 April 2005, the country of birth of the holder no longer appears on the passport. This was the only source of information on the country of birth of NZ citizens travelling to or from Australia. Therefore, with the increased numbers of travellers holding NZ biometric passports, the proportion of movement records with not stated country of birth increased substantially. For other travellers who are not NZ citizens, country of birth information can be obtained from their visa information. Visa information for most NZ citizens is not available as, under the trans-Tasman agreement, they do not need to hold a visa to travel to Australia. For August 2007 data, the total number of not stated responses for country of birth as supplied by DIAC was 76,763. NZ passport holders represented approximately 88% of these non-responses. By August 2008 the total number of not stated responses had increased to 101,684 for the month with NZ passport holders representing 92%.
In order to alleviate this issue, the records with not stated responses for country of birth have been imputed at the category of traveller and country of citizenship level from August 2007. As a result, the total number of not stated responses for country of birth has been reduced to 800 (August 2007) and 745 (August 2008) and the number of non-responses for NZ citizens to zero for both periods. This method of imputation will be in place until a more suitable means of obtaining or imputing country of birth data for NZ citizens can be implemented.
JULY 1998 PROCESSING
Prior to July 1998, the number of overseas-born (excluding NZ) permanent departures of Australian residents was overstated.
In July 1998, DIAC introduced a Box type validation edit to the processing system. This edit checks and corrects the Box type according to the Visa Class/subclass. With the exception of Australian and NZ citizens, only Australian residents departing permanently (Box F) who hold permanent visas are retained in this Box type. For temporary visa holders who incorrectly ticked Box F, their Box type was changed to visitor or temporary entrant departing (Box D). This edit has been ongoing, with over 5,000 records being moved from Box F to Box D in the year 2005-06.
SEPTEMBER 1998 PROCESSING
A problem was experienced in the processing of OAD data for movement dates between 6 September 1998 and 16 September 1998, following the introduction of changes to DIAC's input processing system. This problem may affect in the order of 10% of all September 1998 records used in estimation and result in incorrect details for citizenship, date of birth, sex and country of birth.
PERMANENT ARRIVALS DURING 1999
The number of permanent arrivals during July to December 1999 was revised in October 2000, as advised by DIAC.
SEPTEMBER 1999 PROCESSING
September 1999 overseas arrivals and departures data are revised for movements from, and to, China (excl. SARs and Taiwan) and Hong Kong (SAR of China) in respect of three variables: country of birth, country of citizenship and country of residence/stay. Changes to 'country of birth' and 'country of citizenship' have been made from data supplied by DIAC. Changes to 'country of residence/stay' have been made by assuming the average proportion of country of birth to country of residence/stay for migrants from China (excl. SARs and Taiwan) and Hong Kong (SAR of China) in September 1995 to September 1998.
SEPTEMBER QUARTER 2000 PROCESSING
A processing error was identified which affected the distribution of short-term resident departures by reason for journey for the months of August and September 2000. Affected data were re-processed, and a revised copy of table 3 for the September quarter 2000 was reissued in the supplement October to December 2000 issue of this publication.
OCTOBER 2005 PROCESSING
In June 2006 DIAC advised that some passenger cards, predominantly from Adelaide airport, were not processed in time for the October 2005 Overseas Arrivals and Departures data cut off date. The October 2005 data were reprocessed to include the missing records (approximately 7,800) and the revised data were published in the June 2006 issue of this publication. All associated time series spreadsheets and data files were revised.
Differences between the revised October 2005 data and the figures published in the October 2005 issue of this publication were spread across all categories of movement and all variables. A breakdown of the numeric differences by category of movement is provided in the June 2006 issue of this publication.
NOVEMBER 2008 PROCESSING
DIAC advised that the previous supply of November 2008 data excluded 3,580 arrivals due to passenger cards being mislaid. This represented 0.4% of all arrivals in November 2008. For further details see the November 2008 issue of this publication.
The ABS and DIAC reprocessed November 2008 data and the revised data are released in the December 2008 issue. As well as the missing arrival passenger cards identified above, processing was completed on additional cards and these records were included in the November 2008 revision. The additional identification of records was mainly due to the movement of the monthly processing cut-off date and reprocessing November 2008 data. The total additional movements affect both arrivals (6,144 movements) and departures (4,078 movements).
MAY AND JUNE 2012 PROCESSING
Investigations by the ABS and DIAC uncovered a processing lag in passenger cards for May and June 2012, in particular from Australian seaports. The ABS and DIAC reprocessed May and June 2012 data and the revised data are released in the August 2012 issue. All associated time series spreadsheets and data files were revised.
An additional 9,300 movements were added in May 2012 and an additional 1,850 in June 2012. This represented 0.4% and 0.08% of movements in the months of May and June respectively. The majority of these movements were through Australian seaports. For May 2012 there were an additional 3,970 arrivals and 1,880 departures by seaport. For June 2012, the additional arrivals by seaport were 3,730 and for departures by seaport an extra 200 movements.
As well as the processing of the lagged passenger cards from seaports, as identified above, additional cards had automatically been processed due to the later date being set for reprocessing of May and June 2012. This re-extraction of matched movements data from the TRIPS processing system allows the capture of additional matched records from other lagged cards arriving and the continual data interrogation and quality checks undertaken by DIAC. These additional processing efforts also resulted in an adjustment to airport arrivals data for June 2012, reducing the number of movements by 3,360.
CHANGE TO PROCESSING OF INTENDED LENGTH OF STAY
There is evidence to suggest that when completing the intended length of stay question on the incoming passenger card (Box B), some passengers are entering their arrival/departure date or their birth date rather than their intended length of stay.
From September 2003, a rule has been implemented to the data processing system stating that if all three elements are complete (years, months and days), then the intended length of stay is to be coded to a non-response. The ABS currently assigns 'not stated' duration as a short-term movement, however a review of this procedure will be undertaken in the future.
This procedure changes the prior data processing system which read only the years from the field on the passenger cards. The previous data processing system could have added to overestimation of the number of long-term visitor arrivals.
These documents will be presented in a new window.