APPENDIX DATA QUALITY ISSUES
SCANNING AND IMAGING
The introduction of a new passenger card processing system from July 2001 has meant that information is now available on the frequency and impact of data item imputation. Much of this information has not been available previously. Additionally, the move to a new processing system has also given rise to new data quality issues directly associated with scanning and imaging.
DEFECTIVE CARDS
There are a small number of unreadable or damaged passenger cards for each month. The Australian Bureau of Statistics (ABS) receives a count of these cards from the Department of Immigration and Multicultural and Indigenous Affairs (DIMIA) by movement direction, box type (category of traveller) and port of clearance. The information on these cards is then physically processed by the ABS and included in the estimates presented here.
DURATION OF STAY
From July 1998 DIMIA has been able to determine the actual length of stay for departing overseas visitors and arriving Australian residents which was previously collected from information on intended length of stay supplied on the arrival and departure card by the passenger. This new method has resulted in a change in data distribution with the number of passengers staying for one year exactly declining significantly when compared with movements prior to July 1998.
The introduction of the new passenger card processing system from July 2001 has shown further evidence of rounding to exactly one year in intended duration of stay/travel as reported by visitors arriving in Australia and Australian residents departing the country. To reflect the historical movement patterns, the records with a reported duration of exactly one year are allocated to short-term and long-term. For visitors, 75% of such records are allocated to short-term and 25% to long-term. The ratio is 67:33 for residents departing Australia.
TRIPS AND MATCHING OF MOVEMENTS ON THE SECOND LEG OF TRAVEL
Each month there are records for long-term residents returning to Australia and long-term visitors departing Australia which could not be matched with DIMIA's Travel and Immigration Processing System (TRIPS) records. Records which did not match with a passenger card have been created directly from TRIPS and added to the ABS processing system.
NEW ZEALAND CITIZENS
Under the Trans-Tasman Agreement, New Zealand (NZ) citizens are not required to have a visa to travel to Australia. As a result, on arrival in Australia their visa documentation cannot be used to determine whether they are either a permanent migrant or a temporary visitor, or an Australian resident returning from NZ. DIMIA believes that a substantial proportion of holders of NZ passports tick Box A (migrating to Australia) each time they arrive in the country, causing an overcount of NZ migrants entering Australia.
The following edits were applied to correct the overcounting of NZ migrants:
July 2001 to June 2002
With the introduction of the new processing system from July 2001, DIMIA coded all NZ citizen arrivals who had ticked Box A and had been to Australia previously (based on immigration records) to resident returning (Box C). If these people were visitors previously, this recoding had the effect of incorrectly reducing the number of NZ migrants whilst at the same time incorrectly increasing the number of NZ citizen returning residents. This problem was overcome by moving the NZ citizens that have been changed by DIMIA from Box A to Box C back to Box A.
July 2002 onwards
From July 2002, DIMIA has introduced a new edit system to ensure accurate Permanent Arrivals of NZ citizens statistics. Where the person ticks Box A on his/her passenger card (first arrival as a migrant) the record is verified by checking previous entries and related passenger card records and if the person is previously recorded as a migrant or resident then they will be counted as returning residents. This will result in more accurate recording of NZ citizens who are migrating to Australia as against those who are residents returning.
NON-RESPONSE
A1 NON-RESPONSE RATES PRIOR TO IMPUTATION(a) - July 2005 |
| |
| | Incoming | Outgoing | |
OAD Variables | % | % | |
| |
Citizenship (Nationality) | 0.1 | - | |
Country of Birth | 0.9 | 0.8 | |
Age (Date of birth) | - | - | |
Sex | 0.1 | - | |
Marital Status(b) | 31.4 | 47.1 | |
Category of travel | 1.3 | 0.8 | |
Permanent migrant | | | |
| Previous/future country of residence | (c)39.1 | 10.6 | |
Overseas visitor | | | |
| Intended/actual length of stay | 4.7 | 1.3 | |
| Main reason for journey | 4.2 | . . | |
Australian residents | | | |
| Actual/intended time away from Australia | 0.7 | 4.3 | |
| Main reason for journey | . . | 4.1 | |
Occupation(d) | 5.0 | 5.1 | |
Country of embarkment/disembarkment | 3.0 | 2.8 | |
Whether intend to live in Australia for next 12 months | 22.2 | . . | |
| |
. . not applicable |
- nil or rounded to zero (including null cells) |
(a) Non-response rates are unweighted. |
(b) Not available for Australian or New Zealand citizens. |
(c) Not available for short-term movements. |
(d) New Zealand passport holders contribute to a large proportion of the non-response rate due to unavailable visa data. |
INTENDED LENGTH OF STAY/TIME AWAY FROM AUSTRALIA
Non-response rates are available for these data items from November 1998. For data prior to November 1998, imputation carried out as part of processing by DIMIA has prevented reliable estimation of non-response rates for these two data items.
MAIN REASON FOR JOURNEY
Before the introduction of the redesigned passenger card in July 1998, 5% of short-term visitor arrivals, on average, were recorded as having a reason for journey of 'Other' or 'Not Stated'. This percentage rose to 14% for July, 16% in August and 29% in September 1998 as a result of processing problems. These problems have now been addressed by DIMIA, with the percentage of 'Other' and 'Not Stated' dropping in October 1998 to 8% and 7% in November 1998.
From the January 1999 issue of this publication, published figures (table 6 in this issue) referencing these three months were revised. The revised data were calculated by estimating the number of persons responding 'Other/Not Stated' using past trends for each country of citizenship and proportionally allocating any persons in excess of the estimated 'Other/Not Stated' total amongst the remaining categories. 'Not Stated' rates are now separately available from February 1999 onwards.
STATE WHERE SPENT MOST TIME
For the months of August 1998, September 1998 and October 1998, data entry problems experienced by DIMIA caused an overstatement of the Northern Territory as the main state of stay with a corresponding understatement for the remaining states and territories. In November 1998 these numbers returned to levels more comparable with previous years, with DIMIA indicating that they had instigated data quality procedures to address this issue.
From the January 1999 issue of this publication, published figures (table 11 in this issue) referencing these months were revised. The revised data were calculated by estimating the number of persons indicating the Northern Territory as their main state of stay using past trends and proportionally allocating any persons in excess of these estimates amongst the remaining states and territories.
With the introduction of the new processing system from July 2001, DIMIA has provided the ABS with data on all missing values of state of stay and state of usual residence. These missing values are now imputed.
DATA IMPUTATIONS
Data was imputed for non-response for state of stay/residence. For state of stay, non-responses were imputed at the category of traveller and state of clearance level. Non-response rates for state of stay are presented in the table below:
A2 NON-RESPONSE RATES FOR STATE OF STAY BY CATEGORY OF TRAVELLER(a) |
| |
| July 2005 | |
Category of traveller | % | |
| |
Permanent arrivals - settlers | 6.4 | |
Long-term residents returning | 1.3 | |
Long-term visitors arriving | 2.5 | |
Short-term residents returning | 0.3 | |
Short-term visitors arriving | 5.7 | |
Residents departing permanently | 2.6 | |
Long-term residents departing | 3.2 | |
Long-term visitors departing | 4.1 | |
Short-term residents departing | 1.9 | |
Short-term visitors departing | 7.0 | |
| |
(a) Non-response rates are weighted. |
Non-responses for country of stay and country of usual residence were imputed in two stages. In the first stage, records with country of stay/residence missing were set to country of disembarkation/embarkation if a response was available. In the second stage, for remaining records where country of stay/residence was missing, values were imputed at the category of traveller, reason for journey and country of citizenship level based on responses to other cards within each subgroup. Accordingly, the level of records with data for country of stay/residence not stated has been minimised.
Change in approach to non-response state of stay for long-term visitor departures
A procedure has been applied before prorating of a non-response to state of stay for long-term visitor departures. If a correction to the box marked by a passenger is made (e.g. a visitor marks a resident box), the state of stay recorded in the incorrect box is applied.
Country of stay
Table A3 below presents the percentage of records with country of stay/residence missing as supplied by DIMIA and prior to imputation.
A3 COUNTRY OF STAY/RESIDENCE NON-RESPONSE RATES BY PASSENGER CARD BOX TYPE(a) |
| |
| July 2005 | |
Box type | % | |
| |
A: Migrating permanently to Australia(b) | 39.1 | |
B: Visitor or temporary entrant | 4.6 | |
C: Resident returning to Australia | 7.5 | |
D: Visitor of temporary entrant departing | 5.6 | |
E: Australian resident departing temporarily | 1.5 | |
F: Australian resident departing permanently | 10.6 | |
| |
(a) As on initial data supplied by DIMIA. |
(b) New Zealand passport holders contribute to a large proportion of the non-response rate due to unavailable visa data. |
Table A4 below shows the non-response rates for country of stay/residence following the application of the first stage of imputation.
A4 COUNTRY OF STAY/RESIDENCE NON-RESPONSE RATES BY CATEGORY OF TRAVELLER(a)(b) |
| |
| July 2005 | |
Category of traveller | % | |
| |
Permanent arrivals - settlers | 4.1 | |
Long-term residents returning | 0.9 | |
Long-term visitors arriving | 0.2 | |
Short-term residents returning | 0.7 | |
Short-term visitors arriving | 0.3 | |
Residents departing permanently | 0.7 | |
Long-term residents departing | 0.1 | |
Long-term visitors departing | 0.4 | |
Short-term residents departing | 0.1 | |
Short-term visitors departing | 0.1 | |
| |
(a) Following imputation based on country of disembarkation/embarkation. |
(b) Non-response rates are weighted. |
SEPTEMBER 1998 PROCESSING
A problem was experienced in the processing of OAD data for movement dates between 6 September 1998 and 16 September 1998, following the introduction of changes to DIMIA's input processing system. This problem may affect in the order of 10% of all September 1998 records used in estimation and result in incorrect details for citizenship, date of birth, sex and country of birth.
PERMANENT ARRIVALS DURING 1999
The number of permanent arrivals during July to December 1999 were revised in October 2000, as advised by DIMIA.
SEPTEMBER 1999 PROCESSING
September 1999 overseas arrivals and departures data are revised for movements from, and to, China (excl. SARs and Taiwan) and Hong Kong (SAR of China) in respect of three variables: country of birth, country of citizenship and country of residence/stay. Changes to 'country of birth' and 'country of citizenship' have been made from data supplied by DIMIA. Changes to 'country of residence/stay' have been made by assuming the average proportion of country of birth to country of residence/stay for migrants from China (excl. SARs and Taiwan) and Hong Kong (SAR of China) in September 1995 to September 1998.
SEPTEMBER QUARTER 2000 PROCESSING
A processing error was identified which affected the distribution of short-term resident departures by reason for journey for the months of August and September 2000. Affected data was re-processed, and a revised copy of table 3 for the September quarter 2000 was reissued in the supplement October to December 2000 issue of this publication.
CHANGE TO PROCESSING OF INTENDED LENGTH OF STAY
There is evidence to suggest that when completing the intended length of stay question on the incoming passenger card (Box B), some passengers are entering their arrival/departure date or their birth date rather than their intended length of stay.
From September 2003 a rule has been implemented to the data processing system stating that if all three elements are complete (years, months and days), then the intended length of stay is to be coded to a non-response. The ABS currently assigns 'not stated' duration as a short-term movement, however a review of this procedure will be undertaken in the near future.
This procedure changes the prior data processing system which read only the years from the field on the passenger cards. The previous data processing system could have added to overestimation of the number of long-term visitor arrivals.