Microdata and TableBuilder: Characteristics of Employment, Australia

Enables detailed analysis of employment characteristics

Introduction

This product provides a range of information about the release of microdata relating to employment characteristics.

Microdata are the most detailed information available from a survey and are generally the responses to individual questions on the questionnaire or data derived from two or more questions.

Characteristics of Employment, 2014 to 2020

The Characteristics of Employment survey (COE) is conducted in August throughout Australia and is designed to provide statistics on weekly earnings of employees which shows: the distribution of weekly earnings of all wage and salary earners classified by full-time/part-time workers; and employment characteristics which presents data on trade union membership, fixed-term contracts, independent contractors, working arrangements, and employment found through a labour hire firm or employment agency, all of which can be cross classified by other employment characteristics such as hours worked, industry, occupation and sector of job as well as personal characteristics.

Microdata from the COE survey is released as a TableBuilder product. TableBuilder is an online tool for creating tables and graphs. Before applying for access, users should read and familiarise themselves with the information contained in the TableBuilder User Guide.

Historical microdata, 1998 to 2010

Prior to 2014, microdata relating to employment characteristics was released in a number of Confidentialised Unit Record Files (CURFs). These files are now available in ABS DataLab

Accessing the data

You can use this data in:

  • TableBuilder - online tool for creating tables and graphs.
  • DataLab - analyse detailed microdata

Compare access options to see what's right for you or Apply for access.

Further information about these products, and other information to assist users in understanding and accessing microdata in general, is available from the Microdata Entry Page.

Further information

Further information about the survey and the microdata can be found in the various pages associated with this product, including:

  • Detailed lists of data items are available in the Data downloads section.

Support

For further support in the use of this product, please contact Microdata Access Strategies on 02 6252 7714 or via microdata.access@abs.gov.au.

Data available on request

Data collected in the survey but not included in TableBuilder may be available from the ABS, on request, as statistics in tabulated form.

Subject to confidentiality and sampling variability constraints, special tabulations can be produced incorporating data items, populations and geographic areas selected to meet individual requirements. These are available, on request, on a fee for service basis. For more information, contact the ABS by visiting www.abs.gov.au/about/contact-us or email the Labour Statistics Branch at labour.statistics@abs.gov.au.

Privacy

The ABS Privacy Policy outlines how the ABS handles any personal information that you provide to us.

Data and file structure

Survey methodology

General information about the Characteristics of Employment (COE) survey, including summary results, are available in the publication Characteristics of Employment, Australia (cat. no. 6333.0).

Detailed information about the survey including scope and coverage, survey design, data collection methodology, weighting, estimation and benchmarking, estimate reliability and a glossary can be accessed from the Methodology page of the publication. All published summary tables are available on the website in Excel spreadsheet format.

Data items

The data items included in the Characteristics of Employment TableBuilder are grouped under broad headings and subheadings as shown in the image below. A complete data items list can be accessed from the Data downloads section.

Headings and subheadings

File structure

The underlying format of the Characteristics of Employment (COE) TableBuilder file is structured at a single person level. This person level contains general demographic information such as age, sex, country of birth and status of employment as well details about weekly earnings, working arrangements and qualifications.

When tabulating data from TableBuilder, person weights are automatically applied to the underlying sample counts to provide the survey's population estimates.

Reference year

The COE TableBuilder contains a mandatory field called Reference year to allow for historical analysis. By default this field will be present in any new table as per the image below:

Reference year will be present in any new tables

Individual years can be removed from the table using the data item panel by selecting the required year and removing it from the table as per the image below:

Select individual years in the field of Reference year

However, at least one category (reference period) of the mandatory field must be present in a table for TableBuilder to retrieve data.

Biennial content

The COE TableBuilder contains biennial content, distinguished by odd and even years in the Data items list as per the image below:

Data items and applicable populations

When a data item is placed in a table and was not applicable for a particular reference year, TableBuilder will return a "Not applicable" reference. Where data is requested for multiple years for a biennial item, TableBuilder will retrieve data at the applicable reference year and return "Not applicable" for the year that the data item was not collected.

Not applicable categories

Most data items included in the TableBuilder file include a 'Not applicable' category. The classification values of these 'Not applicable' categories, where relevant, are shown in the data item list in the Data downloads section. The 'Not applicable' category generally represents the number of people who were not asked a particular question or the number of people excluded from the population for a data item when that data were derived (e.g. Status of employment in second job is not applicable for people without a second job).

Table populations

The population relevant to each data item should be kept in mind when extracting and analysing data. The actual population count for each data item is equal to the total cumulative frequency minus the 'Not applicable' category.

Generally, some populations can be 'filtered' using other relevant data items. For example, if the population of interest is 'Employees', any data item with that population (excluding the 'Not applicable' category) could be used.

Zero value cells

Tables generated from sample surveys will sometimes contain cells with zero values because no respondents that satisfied the parameters of a particular cell in a table were in the survey. This is despite there being people in the general population with those characteristics. This is an example of sampling variability which occurs with all sample surveys. Relative Standard Errors cannot be generated for zero cells.

Availability of median earnings data in TableBuilder

For the Characteristics of Employment survey, median weekly earnings are considered to be a more robust measure of centre for earnings data and have been given more prominence since August 2017.

To minimise the risk of identifying individuals in aggregate statistics, a technique is used to randomly adjust cell values. This technique is called perturbation. Perturbation involves small random adjustments of the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released.

The ABS has tested and implemented a new perturbation process in respect of median earnings data to ensure that both the confidentiality of individuals are maintained, and the integrity of medians is better preserved.

Using TableBuilder

For general information relating to the TableBuilder or instructions on how to use features of the TableBuilder product, please refer to the TableBuilder, User Guide.

More specific information applicable to the Characteristics of Employment (COE) Survey TableBuilder, which should enable users to understand, interpret and tabulate the data, is outlined below.

Confidentiality features in TableBuilder

In accordance with the Census and Statistics Act 1905, all the data in TableBuilder are subjected to a confidentiality process before release. This confidentiality process is undertaken to avoid releasing information that may allow the identification of particular individuals, families, households, dwellings or businesses.

Processes used in TableBuilder to confidentialise records include the following:

  • perturbation of data; and
  • table suppression

Perturbation effects

To minimise the risk of identifying individuals in aggregate statistics, a technique is used to randomly adjust cell values. This technique is called perturbation. Perturbation involves small random adjustments of the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics.

The introduction of these random adjustments result in tables not adding up. As a result, randomly adjusted individual cells will be consistent across tables, but the totals in any table will not be the sum of the individual cell values. The size of the difference between summed cells and the relevant total will generally be very small.

Please be aware that the effects of perturbing the data may result in components being larger than their totals. This includes determining proportions.

Table suppression

Some tables generated within TableBuilder may contain a substantial proportion of very low counts within cells (excluding cells that have counts of zero). When this occurs, all values within the table are suppressed in order to preserve confidentiality. The following error message below is displayed (in red) at the bottom of the table when table suppression has occurred.

ERROR: The table has been suppressed as it is too sparse
ERROR: table cell values have been suppressed

Counting units and weights

Weighting is the process of adjusting results from a sample survey to infer results for the total population. To do this, a 'weight' is allocated to each record. The weight is the value that indicates how many population units are represented by each sample unit.

To produce estimates for the in-scope population you must use a weight field in your tables. In TableBuilder they can be found under the Summation Options category in the left hand pane under the applicable level. If you do not select a weight field, TableBuilder will apply 'Person weight' by default. This will give you estimates of the number of persons.

If you are estimating the number of persons with certain characteristics (e.g. 'Number of non–school qualifications completed') the weight listed under the category heading 'Person level weighting' must be used.

Selecting data items for cross-tabulation

The Person level contains a range of data items detailing the characteristics of the respondent including COE, demographic, education, labour force and population variables.

Populations and data items

When adding a data item to a table, an associated population should also be used to ensure correct data is retrieved from TableBuilder. For example, the data item "Whether employment had a set completion date or event in main job" is only applicable to "Employees in main job", so when using this item in a table only the population "Employees in main job" should also be used.

Similarly, if users want to add multiple data items to a table they should ensure that these data items are applicable to the same population group.

Cross-tabulating data items on the same level

Cross-tabulating data from the Person Level with other data items from the same level will produce data about people. For example, cross-tabulating the geographic variable 'State or territory of usual residence' by the 'Hours usually worked in main job' produces a table showing the number of people in each region by the hours that they usually work each week in their main job.

Multiple-response data items

A number of the survey's data items allow respondents to report more than one response. These are referred to as 'multi–response data items'. An example of such a data item is pictured below. For this data item respondents can report all of the days of the week they usually work.

Multiple-response data item

When a multi–response data item is tabulated, a person is counted against each response they have provided (e.g. a person who responds 'Monday' and 'Thursday' and 'Saturday' will be counted once in each of these three categories).

As a result, each person in the appropriate population is counted at least once, and some persons are counted multiple times. Therefore, the total for a multi–response data item will be less than or equal to the sum of its components.

For more information on definitions and concepts that apply to the data items in this file, please refer to Characteristics of Employment, Australia (cat. no. 6333.0) and Labour Force, Australia (cat. no. 6202.0).

Using DataLab

DataLab allows real time access to detailed microdata files through a portal to a secure ABS environment. Using detailed microdata in DataLab allows users to run advanced statistical analyses using recent analytical software.

For information about the data items available on the detailed microdata files, see the Data Item List in the Data downloads section.

About DataLab

Detailed microdata files in DataLab can be accessed on-site at ABS offices or in a secure virtual environment from your own computer. All unit record data remains in DataLab, and any analysis results or tables are checked by the ABS before being provided to users.

For more information, including prerequisites for DataLab access, please see the About DataLab page.

Historical microdata in DataLab

Prior to 2014, microdata relating to employment characteristics was released in a number of Confidentialised Unit Record Files (CURFs). These files are now available in ABS DataLab

For more information about these microdata releases, refer to the following archived publications:

Data downloads

Data files

Previous releases

 TableBuilder data seriesMicrodataDownloadDataLab
Labour Force Survey and Forms of Employment Survey, 2008 Basic microdata 
Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2010 Basic microdataDetailed microdata
Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2008 Basic microdataDetailed microdata
Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2006 Basic microdataDetailed microdata
Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2004 Basic microdataDetailed microdata
Forms of Employment, 1998 Basic microdata 

Previous catalogue number

This release previously used catalogue number 6333.0.00.001.