DataLab

Applications for overseas access are open to: 

• Domestic researchers temporarily located overseas
• Researchers from an overseas entity commencing new DataLab projects
• Overseas organisations collaborating on existing Australian-led projects
• Analysts and discussants

All applications are subject to a comprehensive assessment, including: 

• Review by ABS security and governance teams
• Review by Data Custodians
• Final approval by the Australian Statistician 

Approval is not guaranteed. Each application is assessed on a case-by-case basis. 

Researchers who are on multiple DataLab projects must be approved for each project.

All approved researchers are subject to and must comply with regular auditing, and must promptly report any relevant changes to their circumstances to ABS.

Applications must demonstrate: 

• A clear and justified need for overseas access
• Alignment with the Five Safes Framework and additional overseas risk controls
• Public value and benefit to Australia
• Organisational support, including endorsement from a Responsible Officer or senior leader
• Sufficient information to support a security and risk assessment (e.g. access location, researcher details)

• The assessment process typically takes:
  • 1 to 2 months for Australia-based researchers wishing to access DataLab from overseas
  • 2 to 4 months for Overseas Entities requesting access. These timeframes can vary depending on data custodian approvals and legal work required
• Charges apply for this service. An assessment fee is payable even if the application does not proceed or is denied by the ABS. For details, refer to Charges

To begin an application or for further enquiries, please contact: mydataportal@abs.gov.au

DataLab access incurs an annual charge. This charge is based on the number of analysts with virtual machine access in a project.

The charge covers the typical annual costs of:  

• project establishment and ongoing administration and support
• researcher onboarding (including training and refresher training)
• changes to analysts and discussants within a project
• standard virtual machine access for analysts
• standard software access for analysts (e.g. R, Python, Stata)
• project storage up to one terabyte
• standard output and input clearance.

The ABS is committed to supporting the DataLab service and subsidising DataLab users. We currently use a partial cost recovery arrangement with charges being set to allow the ABS to maintain its service levels as well as deliver critical system and infrastructure enhancements.   

Please note: 

• Individual quotes will be prepared for projects with over 25 analysts.
• An increase in the number of analysts to the next tier will incur an additional charge equivalent to the next tier.
• Accessing DataLab through a discussant virtual machine (VM) will not affect the project tier. Currently, there are no costs associated with using discussant VMs, though charges may apply in the future.
• Significant changes in project scope may result in the establishment of a new project.
• Access to non-standard services are subject to additional charges, please see the non-standard access and services table below.
• Annual charges are due and payable at the start of the financial year for existing/active projects for the whole year, and cannot be pro-rated.
• Annual charges for new projects commencing during the year are pro-rated to the quarter in which the project commences. For example, a project commencing in October will be charged the annual charge and any non-standard DataLab access charges based on three quarters of the financial year.
• Goods and Services Tax (GST) will be applied to charges unless the organisation is exempt.

Additional charges apply for customised data integration services.

Non-standard charges apply to all projects for services that are not within the scope of the annual charge.

• Non-standard charges include access to SAS, Databricks, non-standard virtual machines and storage above one terabyte.
• Non-standard charges for access to Databricks and non-standard virtual machines will be invoiced quarterly based on usage and as per the schedule below.
• Non-standard charges also apply to increased service levels, such as priority clearance and high service level.

For projects and data services covered by the current ABS/Universities Australia Agreement, students, staff and researchers are not individually invoiced for their use of the charged ABS data services. 

For more information refer to the ABS/Universities Australia Agreement.

University projects that choose to progress their access requests outside of the ABS/Universities Australia Agreement will be subject to access charges as outlined above.

Academics looking to submit a grant application should note the following:

• Access Fees: Projects outside the ABS/Universities Australia Agreement will incur standard charges.
• Eligible Expense: DataLab access is allowed under current National Competitive Grants Program (NCGP) guidelines.
• Cost Estimates: Review published fees above or contact the ABS for multi-year estimates, including potential increases and non-standard charges.

ABS International Merchandise Trade (IMT) statistics are subject to additional disclosure controls and consistency checks with published ABS data when releasing IMT data from the DataLab. 

These additional activities incur additional costs. Projects requiring access to IMT data are subject to the following charges:

• $500 (GST exclusive) – per project requesting access to merchandise imports or exports data.
• $250 (GST exclusive) – per IMT output request submitted.

Access to the DataLab from outside Australia is open for applications, see Access outside of Australia for more information. There are additional charges associated with project and researcher applications to access DataLab from outside Australia. This includes domestic researchers seeking to access DataLab while travelling outside Australia, and researchers from overseas entities seeking to access DataLab for a new project. 

Note, an assessment fee is payable even if the application does not proceed or is denied by the ABS. Paying this fee does not guarantee access.

These charges are in addition to the annual DataLab charges and cover costs for the ABS to administer access to DataLab from outside Australia while upholding high standards of security and service delivery.

Charge Activation

Charges apply when a project is formally established, and data access granted.

A project is established when: 

• At least one analyst is onboarded
• At least one data approval recieved

This approach is similar to a subscription model, where billing begins when the service becomes available, even if all datasets are not yet fully delivered and not all researchers are onboarded and ready for access. 

The rationale is that from the point of activation, resources are allocated, systems are engaged, and support is provided to enable access and use of the available data. This ensures timely delivery and allows users to begin working with what’s ready, while remaining aligned with the broader project timeline.  

Note: Researchers must indicate if they would prefer to delay the establishment of their project until all data or all researchers are ready. 

Price Changes

DataLab prices are reviewed annually. Communications will be sent out to project editors and project leads based on communication preferences. Please ensure project contacts are up to date in myDATA.  

Goods and Services Tax (GST)

Prices advertised exclude GST. GST will be applied to charges unless the organisation is exempt. 

Tier Changes

If a project increases its number of users during the year and moves to a higher pricing tier, a catch-up charge will be issued in the quarter additional user access is granted and tier change has occurred, following the initial annual charge.  

If a project reduces their Tier during the year, there is no pro-rata refund for the decrease in tier.  

The quarterly charge shown in the table below does not apply to projects that are looking to have half-year projects (e.g. start Q1, closure in Q2) as projects must be paid on an annual basis. The quarterly split is only to assist in calculating catch up charges during tier increases. 

An example of tier change charges is highlighted below using 2025-26 DataLab Prices.  Please check the Charges for up-to-date prices. 

• Project is a Tier 1 project at the start of the financial year and has been charged $5,500 for the full FY in July and the invoice is paid.
• In September (Q2) the project increases the users from 5 to 7, this means that the project is a tier 2 project from Q2. The new charge will be a combination of charges - at Q1, on Tier 1 charge and Q2, Q3, & Q4 on the Tier 2 charge; bringing the total charge for the year to $11,275.
• In this case the catch-up/ tier increase charge of $5,775 will be issued (This is $11,275 less the initial annual charge of $5,500 - assuming the initial charge is fully paid).
• If a project reduces their Tier during the year, there is no pro rata refund for the decrease in tier.
• If the initial $5,500 is not paid, a new charge of $11,275 will be issued overwriting the initial quote. 

Existing/Active Projects

DataLab standard access is renewed at the beginning of each financial year based on new prices.  

All annual charges are due and payable at the beginning of the financial year and apply for the full year. These charges are fixed and cannot be prorated nor refunded. 

New Projects

During project establishment, charges are prorated based on the quarter in which the project commences. These projects will be subjected to the full year costs in the new financial year to renew DataLab access for their projects.  

Annual charges are not adjusted or prorated when a project concludes or is planned to conclude. This approach ensures adequate cost recovery for the closure, storage, and archiving activities required at the end of a project.

Automatic Closure 

• Projects may be closed based on the estimated end date recorded in myDATA. If an extension is needed, a change request can be submitted through myDATA before the closure date.
• Charges will continue to apply until the project is formally closed and deactivated in the DataLab, regardless of data completeness or usage during the final period.
• Projects approaching the closure date in myDATA will be notified 12 weeks prior to the estimated end date. If no action is taken or if no response is received, the relevant project will be closed. Reopening a project will incur additional fees. 

Manual Closure 

• The ABS has rights to close a project should a project refuse to pay subscription charges.
• A project will be closed upon inactivity of more than 6 months.
• A final warning will be issued 2 weeks prior to closure. If no action is taken or if no response is received, the relevant project will be closed 2 weeks from the final warning being issued.

End-of-Project Access and Reactivation 

• Projects are not typically configured to be paused and resumed after closure, due to current system configurations. Re-accessing archived project data involves significant resource and cost implications.
• In many cases, the effort and expense required to reactivate a closed project are comparable to maintaining the project continuously for the full year. For this reason, careful consideration should be given before initiating closure, especially if future access may be required.
• Reopening a project will incur additional fees, please contact the ABS if you need to re-open a closed project

• Non-standard charges are charges for services that are not covered as part of the annual charge.
• These charges are for supplier costs and additional services provided above the base suite of software and services.
• Databricks and Non-standard Virtual Machine charges are issued quarterly based on reported usage and access to the non-standard service retrospectively. Schedule of charges as below: 

• Non-standard charges commence once a project or user-based non-standard access is granted. This applies even if some datasets are incomplete or not yet available at the time of access.
• The following table summarises non-standard services and its charging bases. These services may not follow a quarterly billing cycle and are billed annually to support consistent service delivery and cost recovery: 

• The Universities Australia Agreement covers standard charges only for nominated projects. It does not include any Data Integration or Non-Standard services such as SAS Licences, Additional Storage, DataBricks, Non-Standard Virtual Machines, high clearance demand and high service. These are separate charges invoiced to the project directly.
• Project leads are offered an option to join the queue to be covered under the Agreement during project establishment. Option chosen during proposal cannot be reverted. Projects that opt to be user funded cannot choose to join the queue later.
• Invoicing will be issued by the accounts department from the ABS Finance Department.
• For more information, please refer to the ABS/Universities Australia Agreement

Fee Calulator

Use the Access Outside Australia calculator below to calculate the estimated cost for your project. Final fees are determined by your ABS representative. The calculator must be downloaded. 

Note: Fees calculated using this tool may not be accurate. 

Types of Access Outside Australia projects – charges in AUD and excl. GST:

Domestic (Australian) Researcher Travelling outside of Australia 

If you're an approved DataLab researcher based in Australia and plan to travel overseas while continuing to use DataLab, a one-off Assessment fee of $1,600 applies on the financial year. 

This fee is charged per researcher and covers: 

• Additional approval steps (support, documentation, and revised agreements)
• A cyber-security assessment
• Setup of secure DataLab access while overseas 

All other standard and non-standard DataLab fees apply per the current price catalogue on the Charges page. 

Scenario 1 

We are an Australian based project with five researchers and is charged an annual fee of $5,500 to access DataLab. If one of us plans to travel overseas for six months and wishes to continue using DataLab during that time, an additional $1,600 Assessment fee applies for that researcher. 

In the year that travel occurs, our total annual fee will be $7,100. In all other years, the fee is $5,500, as per the standard DataLab pricing should there be no more travelling researchers. 

                                               Figure 1 

                            Graphs made using fee calculator

Domestic Entity partnering with Researcher(s) overseas or an Overseas Entity with Overseas Researcher(s) – See Scenario 2 & 3 

If you are an Australian organisation with researchers located outside Australia OR an overseas entity with overseas researchers using DataLab, the charges that apply are: 

• All standard and non-standard DataLab fees apply as outlined in the current ABS price catalogue;
• A one-off establishment fee of $4,100 is charged for each overseas entity involved for a project. This covers the costs of assessing the organisation’s privacy and legal settings, customising undertakings, and obtaining custodian approvals;
• A one-off Assessment fee of $1,600 per researcher;
• An annual top-up fee of $1,900 applies per project to support ongoing activities such as maintaining agreements, whitelisting users, monitoring, auditing, and client support; and
• In some cases, an additional legal fee of $7,600 may be charged—particularly for projects or organisations operating under complex legal frameworks, such as international bodies like the OECD. 

Scenario 2 

A. One overseas researcher 

We are an approved Australian research project with a small team of 1 to 5 researchers. 

Recently we collaborated with a researcher from England. This overseas researcher has just been approved to access DataLab. 

In the first year of this arrangement, we would expect to pay $11,200 for the researcher’s overseas access, which includes: 

• $5,500: The standard annual fee for Tier 1 DataLab projects.
• $1,600: An Assessment fee for the overseas researcher, covering the necessary checks to ensure they meet DataLab’s access and security requirements.
• $4,100: An organisation establishment fee, which applies if the overseas institution hasn’t previously been reviewed by DataLab.
• Legal fees may apply, but only if specific legal agreements or custom arrangements are required. (refer to Figure 2) 

 In subsequent years, the cost reduces to $7,400 annually, comprising: 

• $5,500: The ongoing annual fee.
• $1,900: A top-up fee to maintain access for overseas researchers. (refer Figure 2) 

                                        Figure 2

                       Graphs made using fee calculator

B. Multiple researchers overseas  

We are a small Australian-based research organisation that has recently formed a new partnership with an overseas entity. Our overseas partner has two active researchers who will be contributing to the joint DataLab project. 

In the first year of this collaboration, we would expect to pay $12,800 for its overseas access, which includes: 

• $5,500: This is the standard annual fee that applies to all DataLab projects based on tiers;
• $3,200: An Assessment fee for the two overseas researchers (charged at $1,600 per researcher) to ensure they meet DataLab’s access and security requirements.
• $4,100: An organisation establishment fee, which covers the initial review of the overseas partner (assuming they haven’t been reviewed previously).
• Legal fees may apply, but only if specific legal agreements or custom arrangements are required. (refer to Figure 3) 

 In subsequent years, assuming no major changes to the partnership, the cost is estimated to be $7,400 annually, which includes: 

• $5,500: The ongoing annual fee (+ price changes)
• $1,900: A top-up fee to maintain the overseas researcher access. (refer to Figure 3)  

                                        Figure 3

                     Graphs made using fee calculator

Scenario 3

As an overseas entity,  we have five researchers who need access to the DataLab. we should expect to pay $17,600 in the first year, which includes: 

• $5,500 for the annual DataLab fee.
• $8,000 in Assessment  fees for my five researchers (at $1,600 per researcher), to ensure they meet the necessary access and security standards.
• $4,100 as an organisation establishment fee
• Legal fees may apply, but only if specific legal agreements or custom arrangements are required. 

In subsequent years, assuming no changes in project agreements, annual cost would be approximately $7,400, which covers: 

• $5,500 for the ongoing annual fee.
• $1,900 as a top-up fee to maintain access for my researchers. 

                                        Figure 4

                    Graphs made using fee calculator

At present, ABS do not have the system capability to support prepayment arrangements as part of our standard charging processes. However, for projects with anticipated charges exceeding $5,500, prepayment discussions can be initiated on a case-by-case basis. These arrangements must be agreed upon in advance and are subject to internal approval and operational feasibility. 

Tier 1 and 2 projects (projects with under 10 analysts) 

• Tier 1 and 2 projects continued to be charged the same annual rates for financial years 2023-24 and 2024-25.
• Prices increased from 2025-26 for all Tier 1 and 2 projects.

Tier 3 projects (projects with 11 to 25 analysts) 

• Tier 3 projects were charged half the 2022-23 annual charge of $10,000 ($5,000 excluding GST) for 1 July 2023 to 31 December 2023, with prices increasing from 1 January 2024.
• From 1 January 2024, Tier 3 projects were subject to a new charge rate.
• Tier 3 projects were charged half the new 2023-24 annual charge of $30,000 ($15,000 excluding GST) for 1 January 2024 to 30 June 2024.

Tier 4 projects (projects with over 25 analysts) 

• Tier 4 projects were charged custom pricing from 1 July 2023 under the new pricing model.

ABS International Merchandise Trade (IMT) statistics were subject to additional disclosure controls and consistency checks. In 2024-25 projects requiring access to IMT data were subject to the following charges:

• $500 (GST exclusive) – per project requesting access to merchandise imports or exports data.
• $250 (GST exclusive) – per IMT output request submitted.

Non-standard charges applied to all projects for services not within the scope of the annual charge. Non-standard charges included access to SAS, Databricks, non-standard virtual machines and storage above one terabyte. Non-standard charges also applied to increased service levels, such as priority clearance and high service level. More information on non-standard access and services are in the tables below. 

Detailed microdata files and reference periods in DataLab are listed below. For datasets in other systems see MicrodataDownload and TableBuilder, or all topics in Available microdata and TableBuilder.

You need to apply for access by submitting a DataLab project proposal before you can access these files.

Use Ctrl+F (Windows) or Command+F (Mac) to search this list.

Business Longitudinal Analysis Data Environment (BLADE)

Business Characteristics, 2004-05 to 2009-10, 2006-07 to 2010-11, 2008-09 to 2012-13, 2009-10 to 2013-14, 2010-11 to 2014-15, 2011-12 to 2015-16

Innovation in Australian Business, 2003

Management and Organisational Capabilities of Australian Business, 2015-16

Household Energy Consumption, 2012

Australian Health Survey, Core Content - Risk Factors and Selected Health Conditions, 2011-12

Disability, Ageing and Carers, 2015

Disability, Ageing and Carers, 2018, 2022

Mental Health and Wellbeing, 2007, 2020-21, 2020-2022, 2020-2022 Follow Up, 2021-2022

Mortality, Enhanced Characteristics, 2011-12

National Aboriginal and Torres Strait Islander Health Measures Survey 2022–24

National Aboriginal and Torres Strait Islander Health Survey, 2004-05

National Aboriginal and Torres Strait Islander Health Survey, Core Content - Risk Factors and Selected Health Conditions, 2012-13

National Aboriginal and Torres Strait Islander Health Survey, Detailed Conditions and Other Health Data, 2012-13

National Aboriginal and Torres Strait Islander Health Survey, Nutrition and Physical Activity, 2012-13

National Aboriginal and Torres Strait Islander Social Survey, 2002

National Aboriginal and Torres Strait Islander Social Survey, 2008

National Aboriginal and Torres Strait Islander Social Survey, 2014-15

National Aboriginal and Torres Strait Islander Health Survey, 2018-19

National Aboriginal and Torres Strait Islander Health Survey 2022-23

National Aboriginal and Torres Strait Islander Survey, 1994

National Health Indigenous, 2001

National Health Measures Survey 2022–24

National Health Survey, 2001

National Health Survey, 2004-05

National Health Survey, 2007-08

National Health Survey, 2011-12

National Health Survey, 2014-15

National Health Survey, 2017-18, 2020-21, 2022

Nutrition and Physical Activity, 2011-12

Patient Experiences, 2018-19, 2019-20, 2020-21, 2021-22, 2022-23, 2023-24, 2024-25

Smoker Status, 2017-18, 2020-21, 2021-22

Sun protection behaviours, Nov 2023 to Feb 2024

Australians' Employment and Unemployment Patterns, 1994-97

Employee Earnings and Hours, 2006, 2010, 2012, 2014, 2016, 2018, 2021 and 2023

Employee Earnings and Jobs, 2011-12

Employment Arrangements and Superannuation, 2000

Employment Arrangements, Retirement and Superannuation, April to July 2007

Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2006

Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2008

Labour Force Survey and Employee Earnings Benefits and Trade Union Membership, 2010

Labour Force Survey and Labour Mobility, 2008

Labour Force Survey and Labour Mobility, 2010

Labour Force Survey and Labour Mobility, 2012

Longitudinal Labour Force, monthly data from 1982, includes the following supplementary survey data:

- Barriers and Incentives to Labour Force Participation, Australia

- Characteristics of Employment, Australia

- Labour Force Status of Families

- Participation, Job search and Mobility, 2015-2022

- Work-Related Injuries, 2009-10

Pregnancy and Employment Transitions, 2005, 2011

Qualifications and Work, 2018-19

Adult Literacy and Life Skills, 2006

Australian Census and Migrants, 2011, 2016, 2021

Australian Census and Temporary Entrants, 2016, 2021

Australian Census Longitudinal Dataset, 2006-2011, 2006-2021, 2011-2021, 2016-2021

Census of Population and Housing, 2001

Census of Population and Housing, 2006

Census of Population and Housing, 2011

Census of Population and Housing, 2016, 2021

Characteristics of Recent Migrants, 2007 and 2010

Child Care, 1999

Child Care, 2002

Child Care, 2005

Childhood Education and Care, 2008

Childhood Education and Care, 2011

Crime and Safety, 2002

Crime and Safety, 2005

Crime Victimisation, 2009-10

Education and Training, 2005

Education and Training, 2009

Education and Work, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2022, 2023, 2024, 2025

Family Characteristics and Transitions, 2006-07

Family Characteristics, 2003

Family Characteristics, 2009-10

General Social Survey, 2002

General Social Survey, 2006

General Social Survey, 2010

General Social Survey, 2014

Household Expenditure, Income and Housing, 2003-04 including Fiscal Incidence Study

Household Expenditure, Income and Housing, 2009-10 including Fiscal Incidence Study

Household Expenditure, Income and Housing, 2015-16 including Fiscal Incidence Study

Income and Housing, 2000-01, 2002-03, 2003-04, 2005-06, 2007-08, 2009-10, 2011-12, 2013-14, 2015-16, 2017-18, 2019-20

Person Level Integrated Data Asset previously known as Multi-Agency Data Integration Project (MADIP), 2011-2016

Multipurpose Household Survey, 2004-05, includes the following:
- Household Use of Information Technology
- Barriers and Incentives to Labour Force Participation
- Retirement and Retirement Intentions

Multipurpose Household Survey, 2005-06, includes the following:
- Household Use of Information Technology
- Participation in Sports and Physical Recreation
- Attendance at Selected Cultural and Leisure Venues and Events
- Sports Attendance
- Work-Related Injuries

Multipurpose Household Survey, 2006-07, includes the following:
- Adult Learning
- Barriers and Incentives to Labour Force Participation
- Retirement and Retirement Intentions
- Household Use of Information Technology
- Family Characteristics and Transitions

Multipurpose Household Survey, 2007-08, includes the following:
- Environmental Views and Behaviour
- Household Use of Information Technology
- Personal Fraud

Multipurpose Household Survey, 2008-09, includes the following:
- Crime Victimisation
- Barriers and Incentives to Labour Force Participation
- Retirement and Retirement Intentions
- Household Use of Information Technology

Outcomes from Vocational Education and Training in Schools, 2006-2011

Participation in Sport and Physical Recreation, 2009-10

Participation in Sport and Physical Recreation, 2011-12

Personal Fraud, 2007-08

Personal Income of Migrants, annually from 2009-10, 2010-11, 2011-12, 2012-13, 2014-15, 2015-16, 2016-17

Personal Safety Survey, 2005

Personal Safety, 2012, 2016

Preschool Education, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024

Programme for the International Assessment of Adult Competencies, 2011-12

Time Use, 1997

Time Use, 2006

DataLab safe researcher training is mandatory for all researchers looking to access the DataLab as an analyst or discussant. 

You can seek approval to join a project prior to completing training and onboarding. 

Final approval and access will not be granted by ABS until your training and onboarding is complete.

Training is run virtually and covers:

• researchers shared responsibilities as a DataLab user, why compliance is important and critical for access
• understanding legislative requirements and what that means from a researcher perspective
• understanding conditions of access, researcher responsibilities and obligations
• how to safely use data and prepare appropriate safe output for clearance

The training does not include:

• using the DataLab or myDATA systems
• understanding or using the data available
• code or analytical language training

Safe researcher training and DataLab access are only available to researchers located in Australia in Australian organisations. International researchers and organisations will be considered on a case by case basis. 

Note: If you have previously completed DataLab training, please register your account and reach out to Data Services at mydataportal@abs.gov.au to have your training information added to your profile.

Register for DataLab safe researcher training via the myDATA portal. The myDATA portal is your one-stop shop for all aspects of DataLab training, onboarding and project management.

Once you have created your myDATA portal user account, click on the ‘My onboarding’ tile. You will find an ‘Enrol in training’ button. Click on the magnifying glass to select the training session that best suits you and click ‘Select’. Should there not be a suitable session, click on the 'No session suitable' and we will contact you directly.

Detailed registration steps are listed in the User Guide here.

Email mydataportal@abs.gov.au if you have issues with registering in the portal.

The successful and safe operation of the ABS DataLab relies upon researchers understanding their responsibilities and obligations when accessing the ABS DataLab.

All researchers seeking access to the ABS DataLab, including discussants, must complete the DataLab Safe Researcher training and satisfactorily complete a quiz before they will be granted access to the DataLab.

To retain access to the DataLab researchers must:

• undertake refresher training every two years or as directed by the ABS Researcher Onboarding and System Support
• resubmit relevant Declarations and Undertaking covering your responsibilities and obligations every two years or as required and as directed by the ABS Researcher Onboarding and System Support

If you think you might be due for refresher training please email mydataportal@abs.gov.au

Changing organisations does not invalidate a researcher's training status and the usual refresher training requirements apply.

Refresher training policy:
Researchers need to undertake refresher training because:

• key operations, such as output checking procedures and rules, will change over time
• reinforcing key elements on a regular basis reduces the likelihood that researchers forget them
• people can become complacent about complying with appropriate behaviours in the DataLab
• training reinforces the need to constantly refresh ones skills and knowledge about implementing safe researcher practices
• this ensures researchers remain aware of their responsibilities and obligations when using the DataLab

The refresher training policy requires: 

• all active users and discussants to complete the Safe Researcher DataLab refresher training every two years, or sooner if instructed to by the ABS
• resubmit relevant Declarations and Undertaking covering on responsibilities and obligations every two years, or sooner if instructed to by the ABS
• complete the training in the time-frame specified by the ABS to ensure access is not suspended.  

By accessing and using DataLab, you agree to abide by the Australian Bureau of Statistics (ABS) requirements and obligations, including the conditions outlined below. If you cannot abide by these conditions, your use of the system is to cease immediately.

By using DataLab, I agree:

1. To adhere to all access, usage, security and other procedural guidelines within DataLab as directed by the ABS. Including those provided in the Undertaking by the Responsible Officer of an Organisation, Undertaking by an individual, Declaration of Compliance, and other directions provided to me.
2. To take all necessary measures to protect the security of DataLab and the data held within, by safeguarding my access credentials and promptly notifying the ABS of any security incidents or procedural failures.     
3. To adhere, where applicable, to ABS system constraints.    
4. To the possibility that the ABS may discuss my registration and access with the administering organisation, who have the authority to remove my access.   
5. To comply with the ABS conditions of sale.
6. To use and operate DataLab in compliance with the relevant operating manuals and documentation.   
7. To not remove or attempt to remove content from DataLab by any means, including extracting or copying material by screen capture, handwritten notes, or transcription, without obtaining written approval from the ABS.   
8. To only virtually screen share using DataLab LabLink and only with approved users on the same project.
9. To uphold the integrity of ABS intellectual property by not removing, obscuring, or altering any ABS attributions, including logos, legal notices, or other labels visible in DataLab.   
10. To not attempt to load code, software or applications without seeking the appropriate authorisation from the ABS.    
11. To cooperate with any audit or investigation initiated by the ABS or administering organisation that pertains to any matter concerning the DataLab.
12. Delete or destroy data when requested to do so by the ABS.

I acknowledge that:   

1. My use of DataLab may be audited by the ABS.   
2. DataLab is authorised for use only within Australia unless prior written authorisation has been provided by the ABS.    
3. A breach of these conditions may result in sanctions which may include, but are not limited to, the ABS revoking my access to DataLab permanently or for a set period.   
4. The ABS will not provide guidance on how to conduct data analysis, modelling or how to utilise the statistical tools available.    
5. Features and functionality of DataLab may undergo necessary changes or upgrades without user consultation.   
6. The ABS does not guarantee, or accept any legal liability arising from, or connected to the use of material connected within, or derived from DataLab. 

Expected behaviours: 

1. Comply with the protocols and instructions of the ABS.    
2. Access only the data I have been approved to access. and notify the ABS if you think you have access to data you shouldn't
3. Not attempt to avoid, override, or bypass the system or procedures.   
4. Maintain data confidentiality when submitting outputs for review.   
5. Request output clearance through the ABS DataLab Clearance procedure in all instances.   
6. Notify the ABS of any suspected activities that may impact the security of DataLab.   

Remote access:

Remote access within Australia is permitted under the following conditions:       

1. DataLab must only be accessed from a work or private location.    
2. A secure internet connection must be used.
   • A secure internet connection means any wired or Wi-Fi connection that is password protected (e.g., work, home, your hotel room, hotspotting from your phone).
   • A non-secure internet connection means an open or public connection (e.g., a restaurant/cafe, airport, public transport, hotel lobby or shopping centre).  
3. Overseas access to DataLab is not permitted unless approved by the ABS.    
4. Do not use any type of internal messaging system, do not screen share unless using ABS LabLink and do not transcribe any data from the DataLab prior to output clearance

Please refer to the conditions here for myDATA which are applicable to all users registered in My Data Approvals to Access (myDATA) for access to DataLab projects.

The ABS privacy policy and DataLab user privacy notice  outline how the ABS handles any personal information that you provide to us.   

The ABS Privacy Policy for Managing and Operating Our Business outlines how we handle personal information that is collected for managing and operating within the ABS.  

• encourages, promotes and supports the use of data for research and/or statistical purposes
• provides training on guidelines and compliance requirements for safe researchers and safe use of data
• provides a secure environment for flexible and wide-ranging microdata access to meet researchers' needs
• provides a range of statistical packages and updates
• provides adequate metadata
• manages the authorisation, provision and removal of access to microdata
• provides researchers with the principles and rules for safe outputs
• provides a cultural review of Aboriginal and Torres Strait Islander data projects through its Centre of Aboriginal and Torres Strait Islander Statistics
• checks outputs and provides advice on how to make outputs non-disclosive
• responds to questions relating to the data, processes and systems, in a timely manner
• respects researchers' academic independence
• monitors and audits DataLab use to ensure compliance with procedures and legislative requirements

• submit your research proposal to the ABS by following the steps in the DataLab User Guide
• submit a Change Request in myDATA for changes to the researchers, scope, project time frames and/or data requirements
• support your research team to adhere to DataLab safe researcher practices and behaviours and building a culture of best practice within your team
  • ensure your research team do not share uncleared data or outputs outside of the DataLab secure environment
  • if your research team needs to contact ABS DataLab support areas regarding any uncleared data or outputs ensure these are only stored within your output drive and their location shared in your correspondence
• provide feedback on the outcomes of your project and experience with the ABS microdata and DataLab upon project closure
• provide ABS with two weeks notice before release and then provide a link to any published research stemming from the project's findings
• advise the DataLab team immediately of any suspected incidents in the DataLab, including both data security or procedural failures
• support the ABS in communicating key messages with your research team
• adhere to any relevant requirements of analysts

As an approved project team analyst, you have access to the DataLab and may discuss uncleared data with other approved analysts or discussants on your project team. You must:

• meet all on-boarding requirements, including:
  • completing the safe researcher DataLab training
  • confirming you are willing to have your name, organisation, microdata you have access to, projects, and links to resultant papers published from the research on a register on the ABS website (unless otherwise agreed in advance)
  • confirming you belong to an organisation that has a Responsible Officer undertaking in place with the ABS
  • signing and agreeing to the conditions in the individual undertaking and other associated paperwork (including the Declaration of Compliance)
  • confirming in writing that you are not currently restricted from accessing government data, or any other data due to misuse of data or a breach of data policy/procedures
  • declaring you have at least three years’ quantitative research experience or university study with a significant component working with quantitative data, or if this is not possible comply with the pre-requisite skills and/or experience expected of approved researchers
  • having experience with at least one of the statistical analytical languages available in the DataLab
• comply with ABS protocols and instructions for access and use of microdata in the DataLab
• access only the microdata you have been approved to access - if you can access data that you believe you should not be able to, contact the DataLab team immediately
• inform the ABS if you leave the project team or if you leave your organisation
• only access the DataLab from a private location with a secure internet connection, not from public networks or spaces
• protect your work area and screen from oversight by others, including unauthorised colleagues, family, children and pet cams
• do not screen share unless using DataLab LabLink and only with other approved users on the same project. LabLink is available to all analysts and discussants with a DataLab account
• keep passwords for the DataLab secure
• not share DataLab log in credentials
• not attempt to identify individuals or organisations within data held in the DataLab
• not attempt to match DataLab unit record data with any other list, database or repository of persons or organisations
• not attempt to avoid, override or otherwise circumvent the system or procedures
• not transcribe or copy data, into or out of DataLab prior to formal input or output clearance. Including but not limited to:
  • screen capture, handwritten notes, transcription and/or photographic form
  • only use the shared project space within DataLab to share uncleared work with approved project researchers or to communicate with ABS DataLab support areas about uncleared data
  • only communicate with ABS DataLab support areas about uncleared data via the output drive within DataLab, include only the location of the file shared in your email and if your query involves any specifics of the data this should also ONLY be communicated via the file in DataLab
• do not send outputs to be cleared to the DataLab team - you must request output clearance
• do not attempt to link two microdata files within the DataLab at the unit record level based on ids, record keys or matching characteristics, except where linking keys have been provided and the files are designed to be linked. When the ABS links records to create a linked microdata file, it is assessed to ensure that the confidentiality of the records is protected before being released. Where the linked data is provided in multiple files, they may only be used together if record identifiers have been provided for that purpose and you have been approved to do so as part of your project.
• not deliberately attempt to identify individual or organisational respondents or mishandle a spontaneous recognition event
• be aware that data confidentiality is your responsibility when submitting outputs for review - see Confidentiality in ABS microdata and output guidelines for more information
• report any security incidents or procedural failures to DataLab team immediately via mydataportal@abs.gov.au and cc the lead researcher

Discussants have the same approval as analysts and undertake the same onboarding requirements. 

As an approved discussant by default you do not have access to the DataLab to undertake analysis but may discuss the raw uncleared project data with other approved analysts or discussants in your project team. You can also request a DataLab account to screen share using DataLab LabLink with other approved users on the same project. LabLink is available to all analysts and discussants with a DataLab account.

Discussants must:

• meet all on-boarding requirements, including completion of the safe researcher DataLab training and signing of all relevant undertakings
• must have project approval
• adhere to any relevant requirements of project team analysts

Discussants can:

• view a DataLab session run by an approved analyst on their project, when they are in a secure environment together. For example, together in a secure physical meeting room or secure workstation
• discuss the uncleared data via phone or online meeting (without screen sharing using external software tools)
• screen share via DataLab LabLink with other approved users on the same project

We encourage you to communicate as much as possible within the DataLab environment.

If you need to communicate via other means, consider what is to be communicated and how the communication will take place to ensure that you do not inadvertently remove uncleared data from the DataLab.

Where you may need to seek advice from ABS DataLab support areas, remember that any details regarding uncleared data and analysis can be stored in a file within your Output drive and only the location of this file should be shared in any communication with the ABS.

• Make sure that you only communicate on unvetted data with approved researchers that are approved to work on your project. Unvetted data refers to any data or characteristic of the data. If you learned something from the access you have in the DataLab then you should consider that data and it is not to be shared outside of the system or with unapproved researchers. You can only talk to your supervisor/s about the data, if they are approved researchers or discussants on the project, but consider the environment and who is around.
• Discussions or notes on your project research scope, methods, processes or publicly available metadata may be shared outside of the DataLab (as notes or verbally) and with unapproved researchers.
• The ABS encourages researchers to make notes within the DataLab environment for other approved project team members. You can let them know that you have left a note and would like them to view it. This ensures any data or information about the data remains in the secure system and builds strong safe people behaviours, reducing the risk of a researcher accidentally sending a note outside the system.
• Phone calls and video conferencing may be used for discussions but screen sharing using external software tools is not permitted. Screen sharing is supported using DataLab LabLink. LabLink is available to all analysts and discussants with a DataLab account
• Do not transmit any uncleared DataLab output in an email, including with ABS personnel or the Researcher Onboarding and System Support team. Instead, let your approved project team colleagues or the ABS know and ask them to view the note within the DataLab. Similarly your approved colleagues or the ABS need to leave their responding information within the DataLab and let you know that there is information for you within the project in the DataLab.

ABS trusts and supports approved researchers who remotely access the DataLab.

Remote access is permitted under the following conditions:

• It must be used in a work or private location.
• The screen must be protected from oversight by any other person. This includes password-protecting your screen, should you move away from your computer.
• A secure internet connection must be used:
  • A secure internet connection means any Wi-Fi that is password protected (e.g. work, home, your hotel room, hotspotting from your phone)
  • A non-secure internet connection means an open or public connection like a restaurant/cafe, airport, public transport, hotel lobby or shopping mall
• Overseas access to DataLab is not permitted unless approved by the ABS.
• Working in the DataLab from home is supported by the ABS but you are responsible for checking and complying with your organisation's requirements for working from home.
• Do not use any type of internal messaging system which may have external server connections.
• The DataLab screens are to be kept secure at all times whether you are working within your organisation or from home.

To be an approved DataLab researcher, you must have the analytical research experience to be able to carry out quantitative data research or analysis in the DataLab. This includes the ability to use at least one of the statistical analytical languages supported in the DataLab. This may have been acquired through working on research, analytical or statistical projects. For example, a person who was employed for three years in a relevant field, such as a university researcher, research assistant or a government or non-government employee working in research or statistics. If they had worked for around half of their time on quantitative research projects, then they would have spent a significant component of their time working with quantitative data.

You may also have qualifications (either an undergraduate or higher degree) with a significant proportion of mathematics or statistics. A significant proportion of the degree should cover research method components and analytical fields, including:

• qualitative data collection and research design, interviewing skills, conducting focus groups and ethnographic methods
• quantitative data collection and research design, questionnaire design, sampling and weighting
• hypothesis testing and evaluation
• undertaking systematic reviews
• data analysis, including data linkage, imputation and presentation of results
• application of ethics to research

Other relevant undergraduate degrees may include psychology, demography, social policy, sociology, political science, geography, economics, and social statistics. If you have postgraduate qualifications, you may combine multiple degrees to ensure you meet this requirement. This is a cumulative requirement.

If you do not meet the above criteria but still want to access the DataLab, you may request a referral by an authorised researcher who is on the same research team as you. The referring researcher must meet all of the following requirements:

• have at least three years of either quantitative research or analysis experience or university study with a significant component working with quantitative data
• be working on the same project within the DataLab as the less experienced researcher
• agree to directly supervise and take responsibility for the work of the less experienced researcher
• have the agreement of a Senior Executive from the less experienced researcher’s organisation for this referral

Download the undertaking, declaration and referral forms.

The ABS does not provide support to researchers relating to statistical analytical languages or coding issues.

🗎 Individual Undertaking (U15I)

You need to sign an Individual Undertaking before accessing microdata in the DataLab. The ABS will let you know if an Individual undertaking or other documentation is required under any other circumstances.

The obligations are detailed in full in the Individual undertaking. In summary, you are agreeing that you will:

• only use the information for statistical or research purposes
• not attempt to identify persons or organisations to which the information relates
• not attempt to avoid, override or otherwise circumvent the controls put in place by the ABS
• not attempt to match the information with any other list, database or repository of persons or organisations
• not provide microdata to anyone without ABS approval

🗎 DataLab Declaration of Compliance (U15S1)

A DataLab declaration of compliance is required to be signed by each user prior to accessing the DataLab. By signing the Declaration of compliance you agree to comply with the requirements for accessing microdata through the DataLab, including the requirements of the Responsible Officer undertaking and the Individual undertaking.

You may also need a DataLab referral (please email mydataportal@abs.gov.au for this form). Less experienced researchers must be supervised by another researcher on the same project.

We may request that you also sign other forms for accessing particular datasets, which we will send to you if they are required.

Preparing outputs to be published

• Publishing refers to making information available to the public by any means.
• The ABS encourages researchers to share their research findings (which have been cleared for confidentiality by the ABS) and make the results publicly available.
• If the ABS have cleared and sent you your outputs from the DataLab it means they have passed statistical disclosure checks and are cleared to be released. 

When do I need to inform the ABS I am publishing? 

• Any publication, report and presentation that references BLADE or PLIDA data needs to be provided to the ABS a minimum of 2 weeks prior to wider release. This is a requirement of our Data Custodians. This process does not seek approval from custodians, but rather is in place to give custodians visibility of project outputs, provide comments and brief Ministers, as required. Analysts are sent any comments or feedback provided by custodians on their publications.
• Project teams can use outputs referencing BLADE or PLIDA data as part of preliminary analysis and to collaborate with others. This does not require the 2 weeks notice for the ABS. At this stage your outputs are considered as draft analysis and are not yet published and not for further circulation. When you are circulating this draft analysis, it should be clearly stated the analysis is draft and not for further distribution.
• When your analysis is ready to publish that's when the 2 weeks clearance applies.

Information and research using ABS data must be acknowledged.

When citing the ABS DataLab the preferred citation structure is as follows:

1. Source of Data: e.g. Person-Level Integrated Data Asset (PLIDA).
2. Date/reference period of data used
3. PLIDA product: e.g. PLIDA Modular Product
4. Pathway of access: e.g. ABS DataLab
5. Statement: “Findings based on use of PLIDA data.
6. Include an explanation of any processes or transformations which have been applied to that data

7. If you are using data from ATO, DSS or Home Affairs you must also include the following disclaimer:

   “The results of these studies are based, in part, on data supplied to the ABS under the Taxation Administration Act 1953, A New Tax System (Australian Business Number) Act 1999, Australian Border Force Act 2015, Social Security (Administration) Act 1999, A New Tax System (Family Assistance) (Administration) Act 1999, Paid Parental Leave Act 2010 and/or the Student Assistance Act 1973. Such data may only used for the purpose of administering the Census and Statistics Act 1905 or performance of functions of the ABS as set out in section 6 of the Australian Bureau of Statistics Act 1975. No individual information collected under the Census and Statistics Act 1905 is provided back to custodians for administrative or regulatory purposes. Any discussion of data limitations or weaknesses is in the context of using the data for statistical purposes and is not related to the ability of the data to support the Australian Taxation Office, Australian Business Register, Department of Social Services and/or Department of Home Affairs’ core operational requirements.

   Legislative requirements to ensure privacy and secrecy of these data have been followed. For access to PLIDA and/or BLADE data under Section 16A of the ABS Act 1975 or enabled by section 15 of the Census and Statistics (Information Release and Access) Determination 2018, source data are de-identified and so data about specific individuals has not been viewed in conducting this analysis. In accordance with the Census and Statistics Act 1905, results have been treated where necessary to ensure that they are not likely to enable identification of a particular person or organisation.”

Please refer to these examples:

Example 1

Person Level Integrated Data Asset (PLIDA), 2021, Census of Population and Housing, ABS DataLab. Findings based on use of PLIDA data.

Example 2

Australian Bureau of Statistics (2020) Microdata: Personal Income of Migrants, Australia, accessed 15 December 2020

A citation of your work will be added to the short online description of your project once your work is published, See also How to cite ABS sources

As an approved researcher, you have signed appropriate documentation agreeing to comply with data access provisions under relevant legislation, whenever you access detailed microdata in the DataLab.

If you suspect that you or others in your team may have failed to comply with a microdata undertaking, immediately cease the behaviour, notify the lead researcher and email mydataportal@abs.gov.au as soon as possible.

Where a person is suspected to have breached a microdata undertaking the ABS will investigate and may: 

• immediately suspend their access to the DataLab
• inform the organisation's Contact and Responsible Officers that an investigation is underway
• require that they surrender all microdata in their possession to their organisation's Contact Officer or to the ABS

Following the investigation and depending on the nature of the breach, the ABS may require the development and implementation of strategies to:

• restrict or overcome the consequences of the breach
• ensure no repetition of the breach

Depending on the success of these strategies and the severity of the breach, the ABS may:

• reactivate the suspended access
• extend suspension temporarily or permanently
• suspend or terminate access for the organisation as a whole

The ABS may also invoke Subsection 19 (3) of the Census and Statistics Act 1905 which provides that a person who fails to comply with an undertaking given by that person in respect of microdata is guilty of an indictable offence, punishable on conviction by a fine of 120 penalty units ($39,600 as at 7 November 2024) or imprisonment for two years, or both.
 

While ABS must treat breaches of the microdata undertakings very seriously we would much rather help you and your organisation avoid the possibility of a breach. If at any time, you are unsure about your compliance with a microdata undertaking contact mydataportal@abs.gov.au so we can discuss the circumstances.

To request output clearance, use the clearance request tile in the myDATA portal. 

The ABS must approve and clear all DataLab outputs before you can access them outside the DataLab. Do not copy or remove anything (e.g. data, code, notes) from DataLab yourself. Output requests can take up to 2 weeks to complete. 

Clearance Request best practice:

• Apply all necessary output rules to your analysis and provide evidence of this.
• Create well-organized and clearly described files for a quicker and more efficient clearance process.
• Request only what you need to ensure a more streamlined process.
• Do not include any counts or data from DataLab in your emails with the ABS.

See myDATA for more information.

To request input clearance, use the clearance request tile in the myDATA portal.  

Users can request the addition of aggregate data, concordances, supporting material or statistical code to an existing DataLab project. 

We will not add the following to the DataLab:

• Names of people or businesses.
• Addresses or specific location coordinates (longitude and latitude).
• Large amounts of free text where it is not possible to check for names and other identifying information.

See myDATA for more information.

To request transfer clearance, use the clearance request tile in the myDATA portal. 

Users can move code and files that do not contain data between DataLab projects. Ensure there is no counts or IDs in the files, and all associated log or comment files. 

For data files, provide context, the names of the data products used, and a description for each clearance file requested, including the population scope and definitions of variables. Follow all output rules and provide supporting evidence if required.

See myDATA for more information.

To enquire about the DataLab clearance process, or how to apply output rules, use the clearance request tile in the myDATA portal. 

See myDATA for more information.

The most common types of analysis are listed below along with the applicable rules for output. Other output types will be assessed based on similar principles. 

The rule of 10 refers to the minimum number of contributors required for each cell or statistic. The underlying (unweighted) count of observations must meet this threshold, and evidence must be provided. 

If multiple tables are produced, differences of less than ten should not be able to be calculated through combining the tables. 

The rule of 10 applies to most outputs including counts, percentages (both numerator and denominator), means, sums, ratios, and other statistics. 

Options for making output safe include suppression of small counts, aggregation of categories or perturbation. If a cell is suppressed but it can be derived or estimated from other outputs, one or more additional values should be suppressed to protect the values of the the primary suppressed cell from being worked out.

See Data downloads for examples and options for treatment. 

The dominance rule is designed to prevent the re-identification of units that contribute a large percentage of a cell's total value, which could in turn reveal information about individuals, households or businesses. 

DataLab has a (1,50) and a (2,67) rule. This means that for any cell, the largest contributor cannot account for more than 50% of the total value and the largest two contributors cannot account for more than 67% of the total value. 

Where a variable can take both positive and negative values, the negative values should be replaced with absolute values before determining the largest contributors and the total. The largest absolute value is then divided by the sum of absolute values to determine if the (1,50) rule is met, and the sum of the two largest absolute values are divided by the sum of absolute values to check the (2,67) rule.  

Similar to the rule of 10, in the case of the dominance rule failing and if a cell is suppressed but it can be derived or estimated from other outputs, one or more additional values should be suppressed to protect the values of the primary suppressed cell from being worked out.

Dominance must be checked if any mean, total or similar statistic is calculated for continuous or magnitude variables. It does not apply to counts.

See Data downloads for examples and options for treatment. 

Group (or attribute) disclosure occurs when all or nearly all units that have one feature also have some other feature. This means that even when the individual units may appear protected based on other rules, a previously unknown attribute of a unit may be disclosed based on the attributes of the group. Group disclosure risk should be assessed when any cell contains more than 90% of total number of units in the row or column.   

This rule applies to frequency tables. Whether group disclosure requires treatment depends on the sensitivity and nature of the output. 

See Data downloads for examples and options for treatment. 

Quantiles and other relative ranks must be based on a minimum number of contributors depending on the precision. Underlying unweighted counts should be provided when reporting quantiles in the outputs. For information on required contributors for quantiles, see the table below: 

Minimums and maximums are generally unsafe to output. The following percentiles are safe options if the minimum contributors rule is satisfied: 

• 1st and 99th percentiles
• 5th and 95th percentiles
• 10th and 90th percentiles 

See Data downloads for examples and options for treatment. 

Models and regressions are generally safe to output. However, overfitted models can pose a disclosure risk. All models and regressions must have a minimum of 10 degrees of freedom and evidence that this has been met should be provided.

The degrees of freedom are calculated by subtracting the number of parameters and other model restrictions from the total number of observations that contribute to the model.

See Data downloads for examples and options for treatment.

There are additional rules for specific model types. 

For ordinary least squares regressions, the R-squared should be lower than 0.9. If the R-squared is higher than this, the constant may need to be suppressed to prevent predictions. This requirement does not apply to other models such as fixed effects or two-stage regressions. 

Additionally, for ordinary least squares regressions with a continuous dependent variable and only categorical independent variables, the regression will approximate the tabular means. The addition of a continuous independent variable, or suppression of the intercept reduces the disclosure risk. Otherwise, apply the rule of 10 and dominance rules.

For survival curves, each step change in the survival curve should represent at least 10 data subjects. 

Correlation coefficients should be calculated based on a minimum of 10 contributors.  

Gini coefficients are usually safe to output, and must be based on a minimum of 10 contributors. 

For classification and regression trees, any underlying unweighted counts must meet the rule of 10. 

For other models, please provide evidence that no estimates or parameters are derived from fewer than 10 underlying contributors and explain why the output is non-disclosive.  

See Data downloads for examples and options for treatment. 

All graphs, plots and other charts are subject to the output rules that apply to the underlying output type. The data used in the chart must be provided, accompanied by any relevant supporting evidence that it meets output rules. 

Charts that plot characteristics of individual units or groups of fewer than 10 units will not be cleared. 

See Data downloads for examples and options for treatment. 

Instructions for logging into DataLab:

1. Log into the DataLab portal - access information and settings related to your profile, project and start your virtual machine (VM)
2. Launch the VM - access the project, you and your project team members can view data, run analysis and prepare reports or data outputs for clearance

Note: For security reasons, user accounts will be automatically disabled after 45 days of inactivity and access to your virtual machine(s) will be suspended. The project and all associated data remain unaffected. Notifications are sent by email 14 days and 3 days prior to the account being suspended. You will need to log into the DataLab portal within the 45 days to avoid automated suspension. To reactivate a suspended account, contact mydataportal@abs.gov.au.

Enter your account details into the DataLab log in page.

If you are logging in for the first time, for system security you will need to authenticate your log in using the Microsoft Authenticator app on your mobile phone. To set this up, refer to First time use/new phone steps.

For returning users, click on your account (firstname.lastname@mydata.abs.gov.au) or use another account and enter your account name. All DataLab accounts use the @mydata.abs.gov.au domain format. Enter your password and Sign in.

By logging in you agree to these conditions:

Once you have entered your credentials and hit ‘Sign in’, a notification from the Microsoft Authenticator app is sent to your phone and asks you to perform a “number match”. Enter the numbers shown on your browser screen into the authenticator app on your phone to proceed.

If you don't approve within the time limit, click ‘Send another request to my Microsoft Authenticator app’. If the request expires, re-enter your account and password in the DataLab log in screen.

You can also change the way you approve the sign in request by selecting "I can’t use my Microsoft Authenticator app right now".

You then have two options:

1. approve a request on your phone app
2. use a verification code from your phone app

After you approve the login in the Microsoft Authenticator application, you are logged into the DataLab portal.

To enter your DataLab project workspace you need to:

1. Activate your VM
2. Launch your desktop

Each project VM is displayed on individual tiles, with your “active” VM appearing at the top above those that are “locked”. For more information about your VMs, refer to Functions in My Projects.

Each project has a separate VM and you can only access one project VM at a time. If your machine is already available to launch, skip to Launch your desktop.

• If not, click the ‘Activate' button as shown below and wait until the ‘Connect’ button appears.
• If your machine shows a status other than ‘Connect’ or ‘Activate’ you must rebuild the VM first. Refer toVM management options for more information.

Click on the ‘Activate’ button of the VM you want to launch.

If you have a VM for another project that is currently active, this logs you out of your other session. If you have a program running in your Workspace using another VM, this will stop the program. You can only run multiple VMs if you have requested and are using offline local disk space.

As shown below, you can track the VM activation progress by either selecting 'Track' from the pop up notification or from the action log icon on the left navigator.

When the VM activation completes, an additional pop up notification confirms that the action was successful. If the action fails, repeat the above steps to activate.

If you navigated to the Action Log page, select the laptop icon named 'My Projects' in the left navigator to return to the 'My Projects' page.

Before launching your VM, you can choose your preferred connection. You are required to install the Windows App (for Windows or MacOS) or Remote Desktop client (Windows users only), or have your organisation do so to be able to connect to your VM. All browser-based applications, including Azure Virtual Desktop web client and the Windows App web client are disabled and no longer available for connecting to your VM.

If you are connecting to the 'Remote Desktop client' version of AVD, ensure that your IT department has enabled the correct networking addresses. Your IT department can refer to Enabling access to the Datalab under 'Azure Virtual Desktop configuration' for more information on the required networking addresses.

Note: If you intend to use the 'Remote Desktop client' on your organisation's workspace, you will need to contact your internal IT department to make it available to you.

The latest version of 'Remote Desktop client (MSI)' for Windows, is available here. Ensure you download and install the correct version for your device.

Once you have been provided access by your organisations IT department, or have downloaded, installed and opened the application, click ‘Subscribe’ as shown below.

You will then be shown a new window that allows you to log into your DataLab account using your DataLab credentials. 

Upon a successful login, you will be shown all your available VMs. If you can’t see the VM that you want, it may be dormant and would require a rebuild. Rebuilds need to be completed through the DataLab web portal.

If you have access to multiple projects you may be shown multiple tabs, one for each project as shown in the image below. 

Note: If you wish to use multi-display, right click on your VM icon, click ‘Settings’, and turn off ‘Use default settings’. You should now see options for how you would like to display your VM.

Click into one of the computer icons and you will be taken to a login screen to reconfirm your identity, before access is granted to enter your VM. This may not appear if you have recently logged in, through a similar window. 

The below loading bar should briefly appear. 

If, instead of the second login prompt, you receive a message as shown in the below image, then either your machine is not the active machine or it is not using the AVD version ‘2025’ or similar. If your desktop session does not start, repeat  Step 1 Activate your VM and check that the version at the bottom of your VM says ‘2025’ or similar. 

You will then be presented with another login screen for the VM itself. 

A new window will then open on your computer displaying your VM workspace.

For more information on using the workspace refer to Using your workspace.

Download the Microsoft Windows App from the Microsoft Windows App Store.

Note: If you intend to use the Windows App on your organisation's workspace, you will need to contact your local IT department to make it available to you.

Once you have been provided access by your organisations IT department, or have downloaded, installed and opened the application, select sign in.

Sign in using your DataLab username. Once your Username has been entered, you will be navigated to the ABS DataLab branded log in screen to enter your password.

Once you've successfully logged in, you will be able to see all of your available VM's. If you can’t see the VM that you want, it may be dormant and would require a rebuild. You will be required to connect to the DataLab web portal to activate or rebuild your VM.

Select 'Connect' on the device tile you want to access, noting only the 'active' VM as displayed in the DataLab web portal will allow you to progress into your workspace.

At this point, you may be required to enter your DataLab credentials again to launch the VM, if you have not logged in recently.

When attempting to connect to your VM you may encounter the following error. Note: Check the DataLab web portal to ensure that the VM you are trying to launch is active and does not require a rebuild.

If changes have been made to the VM in the DataLab web portal, you may need to refresh the available resources in the Windows App for changes to take effect.

After successfully Connecting to the VM you will then be presented with another login screen for the VM itself. Again, you will be required to enter your DataLab credentials.

By logging in to DataLab you agree to these conditions outlined on the important notice:

You will then have a new window open on your computer displaying your VM workspace.

Upon shutting down your VM you will receive the following confirmation message. Ensure that you shut down your VM once your analysis is complete to avoid running costs.

For more information on using the workspace refer to Using your workspace.

Download the Microsoft Windows App from the MacOS App Store.   

Use of the DataLab on MacOS is only supported through the Windows App for Mac. Ensure your IT department has enabled the required networking addresses, refer to Enabling access to the DataLab under 'Azure Virtual Desktop configuration' for guidance.  

If the Windows App for Mac isn’t already available in your organisation’s workspace, your IT team may need to enable it.  

Once you have access, open the app, select the + icon in the top right, choose ‘Add Work or School Account’, and sign in with your DataLab credentials. 

Once you've successfully logged in, you will be able to see all of your available VMs from the Devices page. If you can’t see the VM that you want to access, it may be dormant and would require a rebuild. You will be required to access the DataLab web portal to activate or rebuild your VM as this cannot be managed through the Windows App. 

Double-click on the tile for the VM that you want to access, noting only the 'active' VM as displayed in the DataLab web portal will allow you to launch your workspace.  

At this point, you may be required to enter your DataLab credentials again to launch the VM, if you have not logged in recently.   

When attempting to connect to your VM you may encounter the following error. If this occurs, check the DataLab web portal to ensure that the VM you are trying to launch is 'active' and does not require a rebuild.   

If changes have been made to the VM in the DataLab web portal, you may need to refresh the available resources in the Windows App for changes to take effect. 

After successfully Connecting to the VM you will then be presented with another login screen for the VM itself. Again, you will be required to enter your DataLab credentials.

By logging in to DataLab you agree to these conditions outlined on the important notice:

You will then have a new window open on your computer displaying your VM workspace.

Upon shutting down your VM you will receive the following confirmation message. Ensure that you shut down your VM once your analysis is complete to avoid running costs.

For more information on using the workspace refer to Using your workspace.

The DataLab uses two factor authentication to provide a secure log in environment. You need to download the Microsoft Authenticator app to your smart phone to use the DataLab.
Open https://datalab.abs.gov.au and enter your credentials.

The first time you log in, enter the username and password provided to you by the ABS. 

If you are using a new phone, refer to the Contact us page for system support. An ABS system administrator will need to reset your Authenticator. 

Note: all DataLab accounts use the @mydata.abs.gov.au domain format.

If you are switching to a new phone (not new account) you will be given the screen ‘More information required’, click ‘Next’.

This will direct you to Step 1 of setting up your Microsoft Authenticator application. Download the Microsoft Authenticator app to your smart phone from the App Store (for iOS) or Google Play (for Android). Make sure that the authenticator is published by Microsoft, as the ABS DataLab only supports Microsoft Authenticator.

Once you have fulfilled the initial requirements noted for Step 1, proceed by clicking ‘Next’.

You will then be guided to open your downloaded Microsoft Authenticator app and add a ‘Work or school account’.

Once you have selected a 'Work or school account', click ‘Next’ from your browser.

The following screen presents a QR code to scan using your Microsoft Authenticator app. 

Initiate the scanning function on your phone, then hover your phone over the QR code shown on your browser screen.

Once scanning is complete, click ‘Next’ on your browser screen. You will be asked to enter the security number shown on your browser screen into your Microsoft Authenticator app, once complete, click ‘Next’.

After entering the security number and receiving approval from your Microsoft Authenticator application, you will see a ‘Notification Approved’ confirmation on your browser. Click ‘Next’ to proceed to Method 2 of setting up your multi-factor authentication.

Once verified, select ‘Next’ and then ‘Done’ to return to the sign in page. 

Set up a new password for your account. Your password cannot contain your user ID. It must be a minimum of 8 characters and contain at least three of the following:

• upper-case letters A – Z
• lower-case letters a - z
• numbers 
• special characters @ # $ % ^ & * - _ ! + = [ ] { } | \ : ' , . ? / ` ~ " ( ) ;

After updating your DataLab password when logging in for the first time, or having your authentication methods reset you will be prompted to add a second verification method.

The default method is a phone call with automated prompts for you to follow. If you prefer to use a different method, select ‘I want to set up a different method’. From here you can choose either a phone call or text message with a sign in code, or Email. You can skip the setup of a second verification method, however you will be prompted to complete it every time you log in to the DataLab portal.

To proceed with default method, enter your country code and phone number and select ‘Next’.

You will receive a phone call to your mobile device using the phone number entered. Answer the call and follow the prompts provided. Your browser will update with the status of the verification.

After successfully verifying your mobile device your browser will update with 'Call answered. Your phone was registered successfully'. Click 'Next' to proceed.

You will then receive a 'success' page, confirming the details were successfully registered. Select ‘Done’ to proceed to the DataLab portal.

If you forget your password click on the ‘Forgotten my password' link. If you have received a notification to reset your password while in your VM, log out and then click on the ‘Forgotten my password' link. 

Your user ID is populated for you. Enter the characters in the picture, or words in the audio, then click 'Next'.

The next screen takes you to Step One of verifying your account. Choose from the options in the left-hand column.

Enter the verification code you received via email, or the approval notification or code from the Microsoft Authenticator app.

Enter your new password, ensuring it has a minimum of 8 characters and contains at least three of the following:

• upper-case letters A – Z
• lower-case letters a - z
• numbers 
• special characters @ # $ % ^ & * - _ ! + = [ ] { } | \ : ' , . ? / ` ~ " ( ) ;

The following screen is displayed once your password has been reset. Select the ‘click here’ link to sign in with your new password.

When you have successfully logged into your Virtual Machine (VM) instance, your DataLab workspace looks like this:

You can use DataLab in a similar way to using other secure networked systems, where you can securely see, use and share data files, analysis and output with the other members of your project team.

Open File Explorer and click on This PC to see the network drives you have access to:

• Drive C can be utilised to run scripts and create new Python virtual environments, not facilitated through either Jupyter Notebook, JupyterLab or Spyder. Do not use the C drive to save your data or scripts permanently. Though you may be able to temporarily save files there, the drive is destroyed on each 14 day rebuild and any information stored there becomes unrecoverable.
• Drive D can be utilised for running scripts and improved compute performance compared to Project and C drives. Noting the D drive is also destroyed with shutdown (nightly or otherwise) or rebuild. Avoid using this drive to for saving files or file storage. This is not backed up outside of the VM and data cannot be recovered if lost.
• Library (L drive): All researchers can see all files in the Library drive. This is where we upload support information, such as statistical language documentation, ANZSIC classification and general access guides for non-standard products. Files cannot be saved to this drive.
• PodLibrary (Z drive): The Podlibrary drive will house information about data that you have access to in your DataLab projects. Files cannot be saved to this drive.
• Output (O drive): Any output you want the ABS to clear should be saved to this drive. Only members of your team can see this drive. See also Request output clearance. Information is backed up nightly and retained for 14 days. Information in this folder remains unaffected by a rebuild.
• Project (P drive): A shared space for your team to work in and store all your project files, as well as set up and run Python and R scripts. Only members of your team can see this drive. Information is backed up nightly and retained for 14 days. Information in this folder remains unaffected by a rebuild. The default storage is 1TB. You will need to review and delete unnecessary files as your project files grow over time. If necessary, an increase to this storage can be requested via the Contact us page. There may be a cost for additional storage.
• Products (R drive): Access data files that have been approved for your project. However, it is best to use the My data products shortcut on your desktop as this shows you only the datasets you have been approved to access, rather than all dataset short names. Files cannot be saved to this drive.
• Local Disk (X drive): If you have been granted local disk space, this can be used to run jobs on offline virtual machines (desktops). You may want to request this option if you have multiple projects that you are actively involved in. There may be a cost associated with attaching local disk space to your VM. The local disk will only be present if it has been allocated to your VM. To request local disk space contact the ABS via the Contact us page
• ​​​​​​Drive A is not to be used. Information saved here is either destroyed with each nightly shutdown and 14 day rebuild, or has restricted access. Attempting to read or write from Drive A will invoke a group policy error due to access controls. In this case please use the C drive or consult your project lead to request local disk space.

If your network drives do not appear in File Explorer, you can click the 'Refresh Network' shortcut on the desktop. A confirmation message appears when this has been successfully refreshed.

To access the data files for your project, use the 'My Data Products' shortcut on your desktop.

The My Data Products folder displays only the products approved for your project.

Selecting the 'Products' drive shows you the short name of all data loaded to the DataLab. However, if you try to open a file that is not approved for your project you are denied access and receive an error.

To access the latest version of each application in the DataLab, refer to the ‘Manage’ option in the portal to transition to the most recent VM version available. VM versions are updated annually, allowing for a suitable transition period for migration. 

Software can be opened using the shortcuts on your desktop or by using search on the Taskbar.

All researchers have access to these applications in the DataLab:

Windows 11 native accessibility features are available in the DataLab. These can be accessed from the Accessibility folder on the Applications list on the virtual machine or from the Windows lock screen. Any Windows accessibility features not enabled by default can be requested, and will be considered in line with relevant data sensitivities.

Microsoft Word and Excel are not currently available, as these applications require an internet connection, which is not supported in a secure system like DataLab. Libreoffice is the alternative offered in the system, with similar capabilities to Microsoft Office.

Firefox and Edge are available to support access for Databricks and for methods such as Jupyter notebooks in order to use Python/R. These browsers cannot be used to browse the internet.

If you require a specific statistical package not currently available, or have an enquiry regarding software versioning, submit a request using the Contact us page.

Managing your R & Python packages explains how you can manage R and Python packages using the Posit Package Manager shortcut on your desktop.

What is LabLink

LabLink is a new video conferencing tool developed by the ABS for DataLab, now available in its trial (BETA) phase to all users. It is designed to help researchers collaborate securely within their Virtual Machines (VMs), even when working from different physical locations. There are no additional costs involved for projects using LabLink.

LabLink has the following key features:

• Secure Access: Only researchers approved on the same project can join LabLink meetings. External access (outside DataLab) is not permitted.
• Audio and Video enabled: Researchers can communicate with other project users clearly and in real-time.
• Screen sharing: Share your desktop or a selected window with other meeting users.
• Familiar Interface: LabLink works similarly to popular tools like Microsoft Teams and Zoom. You can launch it from the desktop shortcut inside your VM.

Please note the following disabled features for data security:

• Speech transcription
• File sharing
• Meeting recordings
• Instant messaging

Using Lablink

Users can launch LabLink via the desktop icon or at meet.mydata.abs.gov.au in Microsoft Edge within their VM. Both options open the LabLink landing page to join and manage meetings. The page includes a “Help Info” button for common questions, and a LabLink User Guide is available in the Shared Library to support access and troubleshooting. Users will need to coordinate the details of their meetings outside of their VM as the sharing of meeting links has been disabled.

By using LabLink, end users agree to comply with DataLab security policies and conditions of use.

LabLink will remain in BETA while we continue to improve its performance.  For feedback or support, contact: sead.support@abs.gov.au

LabLink for discussants:

Discussants will need to submit a change request in myDATA to opt in to a VM to access LabLink.

Currently, there are no costs associated with using discussant VMs, though charges may apply in the future.

Databricks is available to projects within the DataLab as a non-standard product. 

What is Databricks? 

Databricks is a cloud-based Big Data processing platform which provides users with an integrated environment to collaborate on projects and offers a range of tools for data exploration, visualisation and analysis. Within the Databricks environment, users can:

• Build pipelines for streaming data processing.
• Build and run machine learning tools.
• Create interactive dashboards.
• Take advantage of scalable distributed computing capability.

Project analysts will also have access to the Databricks Academy training subscription (an online library of Databricks training guides), in addition to instruction materials on how to setup the Databricks workspace provided in the ABS shared library. 

NOTE: If you are using Azure Data Lake containers with Databricks, Azure Storage Explorer is available as an alternative to AzCopy to manage and transfer your files between your file share drives (output, project, etc) and blob storage. Refer to the 'Azure Storage Explorer User Guide' in the shared library drive.

NOTE: The project datalake storage does not currently have back up due to implementation restrictions, therefore we highly recommend regularly copying your files from the project datalake storage to your project file share storage (your P:/ drive).

How do I allocate a Databricks workspace to my project? 

To allocate a Databricks workspace to your project, you will need to submit a request to mydataportal@abs.gov.au. Once your project is allocated a Databricks workspace, it can be accessed from within your VM using the installed Edge or Firefox browsers.

How will costing work? 

Access to Databricks will be per project and charged quarterly based on usage. Projects will have the flexibility to select between a low or high usage profile. Selecting the appropriate usage profile is determined by how much compute resources project analysts are estimated to consume. The same level of service is applicable across both profiles.

As Databricks uses separate compute power, projects requesting access to Databricks should consider if they need to continue to maintain their existing VM sizes. The option of scaling down the size of existing VMs provides users the opportunity to save on project costs. 

NOTE: The ABS provides information on appropriate Databricks cost management for end users within the shared library (L: Drive).

What are the cluster policy arrangements? 

User analysts can be provisioned with the following cluster policy options: 

Databricks cluster policies will restrict the type and number of workers you can provision for a cluster. If an existing policy does not fit your requirements, you can request a new policy via the ABS. All information regarding this can be found in the ABS shared library.

To ensure the security and integrity of the DataLab, clients will not have administrative access to the Databricks workspace and some usage restrictions may apply. Administration will be exclusively managed by the ABS, aligning with the specified usage restrictions of the DataLab. 

Please contact mydataportal@abs.gov.au with any questions.

If you are working with a specific set of R and/or Python packages, you can now manage these using the Package Manager shortcut on your desktop.

In the Package Manager, click 'Get Started' to navigate to the available packages. You can use this tool to search for packages (in the left column) and install the packages you want to use for your project. If the packages you need are not listed, you can request them using the Contact us page.

The following describes how to access different, older, versions of packages from Posit depending on the language. If you require a new package, newer or older version of a package from CRAN for R or PyPi for Python, then please go to Contact us (from the 'Request new Software or Software packages' form) and the package will be made available subject to availability and security checks. Packages not from CRAN or PyPi will not be available through Posit and these will be subject to a separate security screening processing before they are approved for access in the DataLab if the process is successful. 

To access different package versions in R 

These files can be downloaded from Posit using your chosen R environment through the ‘devtools’ package e.g., ‘devtools::install_version(“your package”, version=“your package version”)’. You will be able to see the available package version for R at the bottom of the Posit page for that package. An example for the ‘dplyr’ package can be seen below.

To access different package versions in Python 

Accessing older python package versions can be done when installing the package through ‘pip’ e.g., ‘pip install pandas==2.1.0’. The available versions of each of the python packages can also be seen at the bottom of the relevant Posit page. An example for the ‘pandas’ package can be seen below. 

Virtual machines, or VMs, are the virtual workspaces you use to undertake your analysis in the DataLab. VMs are created by the ABS as part of the project establishment process, described in About DataLab.

You have one VM for each project. This is a design feature to prevent data from one project being accessed by another project. You can run analysis on multiple virtual machines at the same time, but only if you have been granted local disk space. See Run jobs on offline VMs (desktops). You may want to request this option if you have multiple projects that you are actively involved in.

The ABS offers standard and non-standard VM sizes. Standard VMs are included in the DataLab annual fee, whereas non-standard VMs are subject to additional charges as they are more expensive to run. For more information on charges, see DataLab charges.

Researchers may request access to a non-standard machine for performance or productivity purposes. If you require a non-standard machine, you will need to consult your project lead and your project editor will need to submit a change request in myDATA. Please refer to Change Requests in the myDATA user guide.

Currently offered VMs and approximate running costs are listed in the tables below.

Standard virtual machines

Large VMs are provided as the default and most projects operate efficiently with this size.

If you have a small or medium machine, it can be upgraded to a large at no additional charge. Please contact mydataportal@abs.gov.au for further assistance.

Non-standard machines are available on request and charged quarterly based on usage.

Note: VMs continue to incur running costs if the VM has not been stopped, even when not in use. The ABS recommends researchers shut down their machine during periods of inactivity to avoid unintended charges. Disconnecting, or closing your machine window, is insufficient. VMs are required to be shut down completely as per the VM Management Options to avoid continued running costs.

If you have any questions or require further assistance, please contact mydataportal@abs.gov.au. 
 

The following specialised VMs (also non-standard) capable of supporting machine learning and high-performance computing can also be requested, however these are assessed on a case by case basis with the appropriate justification, and are subject to quote. If the required VM is not listed, the ABS may be able to provide a customised option at an additional charge, please be sure to describe why the available machines do not meet your needs in any justification provided. A list of virtual machines by region can be viewed via the Azure website.

Assigned names of VMs are unrelated to Azure naming conventions. ABS review our provided VM options periodically, please revisit this page for any updates.

If you need to close your DataLab session but want to keep your analysis running, you can sign out or disconnect from the VM. Your programs will keep running until 8pm local time. If you have extended the shutdown time with the ‘Bypass shutdown’ option the scheduled shutdown will occur up to three days later, at the same time.

To sign out of your workspace, click on the button at the top right of your window that shows your initials, then select ‘Sign Out’.  

To disconnect from your session, first click on the Start menu and then select the power symbol. An option to ‘Disconnect’ will appear. Selecting this will immediately disconnect you from the session.  

Please note that the session window will remain visible until you close the tab.

If you want to close your session and end all programs you have running, simply shut down your DataLab session.  

To shutdown your session, first click on the Start menu and then select the power symbol. An option to ‘Shut down’ will appear. Selecting this will immediately shutdown your session. 

The DataLab portal is where you will find information about your DataLab account, projects and virtual machines (VMs).

The DataLab portal displays information across three tabs: 

My Projects

From this tab you can activate, start and launch the VM associated with your project. 

My Account

Use this tab to view your personal contact information and basic account settings. 

Action Log 

Keeps a record of your portal actions. This can help you manage your sessions and provides useful information if you encounter problems with the system. It includes:

• starting VM
• stopping VM
• changing your Active VM
• restarting VM
• rebuilding VM

Left navigator menu

The left navigator menu contains shortcuts that can be used to navigate between pages. Click the arrow to collapse or expand the navigator menu.

Global links

The links at the top right are available from all pages of the portal:

• What’s New acts as a global information centre for the DataLab, showcasing information about new features and updates
• DataLab Workspace to access your VM. You must activate your VM before you can use this shortcut
• About the DataLab to access this user guide
• DataLab Privacy Notice
• User Icon displays your details, including name, user name and user role. This is also where you log out of the DataLab

Your obligations and management responsibilities

The Using DataLab responsibly pages will help you understand your obligations and management responsibilities to handle microdata safely. Read through these pages or Contact Us if you would like any help understanding your responsibilities.

Important messages banner

This banner appears at the top of your DataLab portal window when we have an important message for your consideration or action. Banner messages are only visible in the DataLab portal and will not appear in the Windows Remote Desktop client or Windows App.

You must connect to your Virtual Machine (VM) using Azure Virtual Desktop (AVD). 

In the 'Manage' menu you will be able to switch your VM version to a new version if there is one available.

Users require the Windows App for Windows or MacOS, or the Remote Desktop Client for Windows (MSI) to access their VM. Connecting via the Windows App web client or the Remote desktop web client are disabled and no longer available for connecting to your VM.

If your VM is dormant, no management options are available.

The contents of each pane on this screen are outlined as follows:

Power State: You can start, stop, or restart the VM. please consider stopping your VM when you have finished work, this may save on costs if you are using a non-standard machine. 

Scheduled Shutdown: VMs are automatically shutdown every night at 8pm local time. If you have a program running that you expect to run past 8pm, you can choose to extend your session for up to 3 days by selecting 'Bypass shutdown'.

Scheduled Rebuild: VMs are automatically destroyed and rebuilt every 14 days for security and maintenance purposes:

• you cannot extend this time, however you can choose to rebuild before the schedule time by selecting ‘Rebuild Now’
• it displays a date and numbered count down on the coloured bar, with time adjusted to your local area
• the coloured bar changes, starting with green, moving to orange, and finally red as you get closer to the rebuild date
• after rebuilding the countdown resets to 14 days and allows you to bypass the nightly shutdown
• if you try to bypass a shutdown when your machine is scheduled for a rebuild, the system will deny the action, but offer to ‘Rebuild Now’

If you are an analyst who works across multiple projects, you can request local disk space. This will enable your VM to run jobs offline, noting the 14 day rebuild still applies.

Datasets are stored on a remote file share. Only the 'active machine' has network access to this location. Your 'locked' virtual machines do not. To run offline jobs, you need to request local disk space to be attached to your machine. There may be a cost associated with this.

When running jobs offline, the inactive machine can continue to run your program as it still has access to the data since it is no longer using the remote file share. However, working like this does not allow your project team to see your analysis or output. You should always move your output back to your Project or Output drives where your project team can access and review the output. See Using your workspace for more on the available drives in DataLab.

Pending request to your administrator, Local disks are SSD’s that are attached to the VM as a non-standard product. These drives provide local fast storage for more compute capacity. When attached, these can be identified as the X: drive in the VM. The performance of these drives is comparable to that of the C: drive and are more efficient than the Project or Outputs drives, offering higher input/operations per second (IOPS). Please note the important functionality of this drive for intermediate compute and storage purposes, rather than ongoing file storage.

To use local disk space:

1. Request access to a local disk for your project through the Contact Us page
2. Copy the data products you need to the local disk.
3. Execute your script from the local disk and start the job.
4. In your program, save your output to the local disk.
5. Exit your VM and return to the DataLab portal to activate another machine.
6. After you have finished running your analysis offline (local disk) move your analysis and output back to your Project drive.

Note: The X: drive is temporary storage and any data stored will be deleted on rebuild, deletion of the VM, or project closure whichever comes first. X: drives are not backed up outside of the VM, and data cannot be recovered from the drive if lost. Due to this, the ABS still recommends saving all data in the project and output drives.

Select the ‘My Account’ tab from the navigation panel to see details about your account.

Basic attributes displays your name, email, phone etc. If your personal details are incorrect, please Contact Us with the correct information.

Account settings allows you to opt in or out of receiving email reminders. These reminders let you know when your virtual machine will shut down. Notifications are sent at 4pm AEST/AEDT, prior to the 8pm local time scheduled shut down if you have started your VM that day. It will also remind you before your 14 day VM rebuild. You can change this option at any time by clicking ‘Edit’.

DataLab portal is accessed via a web browser, DataLab virtual machines are accessible via Windows App or AVD Remote Desktop Client. It is recommended the latest version of either of the following are used:

• Chrome
• Microsoft Edge
• Firefox
• Safari

Internet Explorer is not recommended.

Refer to Logging into the portal and workspace for more information on accessing the DataLab portal and connecting to your VM through the Remote Desktop Client (MSI) or Windows App.

Note: Mobile devices are not supported/enabled for the DataLab.

If your DataLab account name is not recognised by your authenticator application, it may be because you have downloaded an authenticator not published by Microsoft. The ABS DataLab only supports Microsoft Authenticator. 

You will need to download the Microsoft Authenticator application to your smart phone from the App Store (for iOS) or Google Play (for Android) to fulfil the login sequence.

The following authenticator applications (similar in appearance to the Microsoft Authenticator) will not function with the ABS DataLab:

If you have verified you are utilising the correct application, but are not receiving prompts to authenticate on your mobile device, check your battery optimisation is turned OFF by following these steps. 

1. Open Microsoft Authenticator, and refer to the top right-hand corner, if there is a red dot above the three white dots, press this and then select 'Allow' to turn the battery optimisation off.
2. Then try logging in again on the DataLab landing page.

If you receive the ‘Error communicating with server’ message on your mobile device:

• If your device is connected to the home WiFi, then please turn your phone over to mobile data and try logging in again.

If you are switching to a new phone/tablet device: 

• the operating system of some mobile devices may not interact as expected with Microsoft Authenticator (namely the iOS 16, and some older versions of Android), in this case, try setup another phone or tablet device. In the case that you still have your old phone it may help if the old account is removed from the Microsoft Authenticator application on the previous device.

Other troubleshooting steps you can follow if you are having issues authenticating are:

• Ensuring your phone is connected to the internet (if required switch from WiFi to mobile data to ensure a good connection)
• From your phones settings menu, ensure push notification are enabled
• Restarting your phone
• Ensuring your have the latest Microsoft Authenticator app version installed
• Clear your phones cache. On iPhone you will need to delete and reinstall the app. For Android navigate to the App storage settings, select 'Storage' and tap 'Clear cache'.
• Ensure your phone is on the latest OS version

• Your account may have been disabled. Accounts inactive for 45 days are automatically suspended to maintain security. Contact mydataportal@abs.gov.au for assistance.
• If you have entered your user name or password using copy and paste, you may have accidentally included hidden characters or a space.
• Your organisation firewall may be blocking access. Try accessing DataLab while disconnected from your organisation's network.
• The ABS DataLab only supports use of the Microsoft Authenticator app.
• If you have changed your mobile phone we need to reset your Microsoft Multi Factor Authentication, email mydataportal@abs.gov.au
• If you need to reset your password this must be done via the Forgot my password link in the initial DataLab sign in screen.
• Clear your browser cache.
• Try a different browser. See Recommended browsers.

DataLab is enabled by cloud infrastructure, which may be blocked by some organisations’ firewall settings.

ABS cannot make changes to external organisations' infrastructure. Project Leads need to supply the information below to each organisation participating on this project.

Network/IT Security sections in each organisation need to review and make changes to authenticate access.

1. Enable authentication to the tenant

Users need to authenticate to one of ABS Azure Active tenants, which may be strictly controlled by government agencies and academic workplaces. Authentication must be enabled to the tenants:

• mydata.abs.gov.au
• absmydata.onmicrosoft.com

2. Allow user access to URLs

Users will need to access the following URLs:

• DataLab production portal: datalab.abs.gov.au and sead.abs.gov.au 

3. Configure your organisation's network to allow outbound connections to the following addresses required for Azure Virtual Desktop (AVD):

• login.microsoftonline.com
• *.wvd.microsoft.com
• *.servicebus.windows.net
• go.microsoft.com
• aka.ms
• learn.microsoft.com
• privacy.microsoft.com
• query.prod.cms.rt.microsoft.com  

These addresses all utilise the TCP protocol and outbound port 443 for communication. Contact mydataportal@abs.gov.au for further assistance. 

The DataLab has more functionality and features available to you, so you can set options as well as undertake your research.

• First log-in is to the DataLab portal, where you can view and set options for your DataLab account information and virtual machines. Read more in DataLab portal features.
• Second log-in is to the DataLab workspace where you undertake your analysis.

• The temporary password issued to you by the ABS lasts for 90 days. After you have completed the set up steps you must reset your password.
• If you have forgotten your temporary password, email mydataportal@abs.gov.au for a reset.

Your log in credentials for the DataLab portal are the same as for the DataLab workspace. You can reset your password by clicking on the Forgot my password link.

Your session will continue on until a shutdown is required (either nightly shutdown or 14 day rebuild). However, you can still reset your password while your session is running.

1. You must Activate, then start the VM. Follow the process and wait for each step to complete before progressing.
2. Check your internet connection. If you have a weak or intermittent connection, this can affect launching your virtual machine.
3. Try launching the virtual machine outside of your organisation's online environment. Some institutions’ or Government departments’ firewall or other security settings may be preventing access to DataLab portal and/or launching of the VM. Attempting to connect outside your agency’s online environment may assist in forming the VM connection.
4. Restart your virtual machine. As with restarting a computer, restarting your virtual machine can sometimes resolve problems with launching your machine successfully. From the virtual machine page click the Restart VM button and wait 10 minutes to ensure the reboot of the machine is complete before attempting to launch again.
5. If you are still having trouble, email mydataportal@abs.gov.au.

If you are a member of multiple projects in the DataLab, you will have more than one virtual machine. Your Active machine is the one that is connected to the remote file share, where the data files are stored. For security purposes, only one of your sessions can connect to the remote file share at a time (this is where data files are stored). You can activate your virtual machine by using the Change Active VM button.

If you receive the following error or similar to the image directly below when attempting to launch your VM from the Remote Desktop client. Check that your VM is using the latest VM version, that the machine you are trying to launch is ‘activated’, and that your machine is not currently being rebuilt.

To use multiple displays when using DataLab, on the Remote Desktop client (MSI) or the Windows App, you will need to adjust the display settings.

To access the display settings, on the Remote Desktop client right click on the machine you want to use multiple displays for and click ‘settings’, then turn off ‘Use default settings’ to see the display settings.

On the Windows App for Windows, select the three dots in the bottom right hand corner of the VM tile, select ‘Settings’, then turn ‘Use default settings’ off. You will be able to change the display settings under 'display configuration' to allow the use of multiple displays.

When launching your VM through the web portal using the 'Connect' button, the VM window may open but with issues relating to the screen resolution. You can try launching your VM directly from the Remote Desktop Client or Windows App. Refer to Logging into the portal and workspace, then connect ‘Remote Desktop for Windows’ section. 

If required, you can modify the display settings for the VM you’re wanting to launch. From the Remote Desktop Client or windows App, open the display settings and turn ‘use default settings’ off. This will allow you to see what display settings are enabled. Ensure the option ‘Fit session to window’ is enabled.

Microsoft does not currently offer an AVD client for Linux users. Because of this, we are unable to provide support for Linux users looking to access DataLab. While other options for connecting to the DataLab using Linux may exist, for example setting up a Windows VM on your Linux computer, then connecting to your DataLab VM using the Windows App or Remote Desktop Client, we cannot guarantee its reliability.

Virtual machines are destroyed approximately every 14 days for security purposes. If the 14 day timing will interfere with the timing of your project, you can choose to destroy and rebuild earlier than 14 days at a time that suits you.

Virtual machine project and output drives are backed up every night and kept for 14 days. Files outside of these drives are not recoverable.

Save your work to your Project or Output drives to ensure that your analysis is not lost. Information saved outside of these drives is destroyed when your machine is rebuilt every 14 days.

Only if you have requested local disk space to be allocated to a machine. This allows you to run jobs on offline VMs.

Try logging out of and stopping your VM, then begin the Start VM process again. If that does not work, try the rebuild now from your VM management options.

The upload button that is present in your toolbar is intentionally non-functional. You may encounter a success message after uploading but you will not be able retrieve any files that you have uploaded. If you wish to upload data or packages to the DataLab, please contact the Data Services team or Input Clearance team.

When accessing the DataLab through your organisation's virtual machine it is expected that the access would be slow given the multiple levels of virtualisation and processing of inputs. We recommend accessing the DataLab on your base machine, which may make your VM more responsive.

System performance issues can occur for many different reasons, and every issue requires a unique approach to troubleshoot. Many issues can be resolved by attempting one of the following:

• Ensure you have a good internet connection
• Close and reopen the program you are using, and close any other programs or processes that may be consuming system memory
• Confirm your project drive has available space, if not attempt to free up space
• Refresh your network drives using the icon on the workspace desktop
• Attempt to shutdown or rebuild your virtual machine

If issues persist, email mydataportal@abs.gov.au for further assistance.

If you cannot see the Library, Project, and Output network drives in File Explorer, go to the desktop and double-click the Refresh Network Drives icon.

Stata error example

This means you have exceeded the memory for your virtual machine.

1. Use an alternative method/program to manipulate or process the dataset. Some processes/programs/methods for working with large datasets are more memory-intensive than others. Try some alternative method to see if it is less system intensive.

• Most statistical software tools are able to filter data as it is imported. If your analysis only needs variables a, b and c from a dataset containing 30 variables, then selecting, filtering or importing only these variables uses less memory.
• If you cannot do this in your software, consider creating a subsetted data file using another tool, such as Python, as the first step of preparing your data for analysis.
• If you are unsure of alternative methods, we recommend discussing with other researchers in your project team who are more familiar with your chosen statistical software. The ABS does not provide advice or training on using the analytical tools provided to you in the DataLab.

2. To request a larger machine, please submit a change request in the myDATA portal. Larger machines incur higher running costs. With user charging, you may need to consult with your organisation to confirm incurring additional expenses for your project before applying for a larger machine.

Users with SAS installed now have the ability to edit the SAS configuration file located at “C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg”. This file governs various software settings and parameters, enhancing customisation options. Noting any changes will need to be repeated following machine rebuilds, as the C drive is destroyed on rebuild. To modify the SAS config file, follow these steps: 
1.    Access the file at the provided directory, using SAS or a text editor. 
2.    Proceed to make the necessary changes, in order to tailor the configuration.
3.    Save the file (and a backup version on your P drive, to reinstate following machine rebuilds), then test the software for desired outcomes. Exercise caution, as improper edits may lead to unexpected behaviour. 

DataLab users can access a pre-written SAS configuration script that enhances SAS efficiency within the DataLab. This includes the "VM_COMPLETE_SETUP.bat" script and supporting resources to streamline the setup process following machine rebuilds. Please refer to the resources located in the Shared Library Drive (L: Drive).

Clean up the drive contents, review and delete redundant files to free up space.

To request a storage increase, please submit a change request in the myDATA portal. There may be a cost associated with this.

You can request input clearance for data, code or files to be loaded to your project, from either another project, or other sources that you hold.

Yes, each virtual machine has R, STATA and Python as default software. SAS is not automatically provided on all machines but can be requested as it requires a licence to be assigned to your virtual machine. To request SAS, please submit a change request in the myDATA portal. 

When launching Jupyter Notebook or JuypterLab from Powershell or command prompt may be looking in the incorrect directory resulting in an error message. To overcome this, enter the following command into Powershell or the Command prompt:

• To launch Jupyter Notebook, enter: 'python -m notebook'
• To launch JupyterLab, enter: 'python -m Jupyterlab'

Python packages and their dependencies are regularly scanned for known vulnerabilities. Any package versions that exceed the vulnerability threshold will not be available in Posit. If this occurs, check if there are alternative versions of the packages available. In most cases other versions of the package will be available to address known vulnerabilities. If you require assistance, please contact your administrators.

Yes, it takes about 5 minutes to process the connection. You also need to log out of your virtual machine to allow the system to refresh your session with the new data.

You can extend your session to bypass the nightly shutdown, by one, two or three nights.

Use the R Studio Package Manager shortcut on the DataLab virtual machine desktop to check the range of R packages available to you. See Managing your R packages.

By default, packages are installed onto the C drive which gets wiped every time the VM rebuilds. To have packages which persist through rebuilds, we recommend changing the file path to have packages installed on either the Project drive (P:) or a local disk (if attached). Files saved to these drives are retained over rebuilds.

You can check for any background processes that can be turned off for RStudio. For example, turning off the “show diagnostics for R” may improve the performance. To do this, go to tools, then global options, and click on 'code'. Under diagnostics untick "show diagnostics for R", then select Apply and Okay.

If the project you opened was saved with SAS Datalab – [machine name] you are connecting to the local SAS server without a profile. When you try to run the project without selecting a profile the system may present an error message saying "The server "SASMain" is not defined in the current repository". Click though the messages and continue.

1. See Managing your R packages to use the RStudio Package Manager on the desktop.
2. If the packages you need is not listed, email your request to mydataportal@abs.gov.au

Due to a default setting in Microsoft, the system automatically uses Microsoft Edge to open any PDF file. You can open the PDF file by right-clicking on the file, selecting Open with > Adobe Reader. This launches the file using Adobe Acrobat Reader.

Following recent enhancements, PostgreSQL's data directory relocation is now automated to X:\psql\data when a local disk is attached, facilitating expanded storage capacity for data. This feature streamlines the process of accommodating larger datasets. To enable local disc please email mydataportal@abs.gov.au

When creating a new compute cluster in Databricks there is a known issue with the new ‘Simple’ compute creation form. Ensure that the Simple form setting is switched to ‘Simple form: OFF’ when creating a cluster. This will enable you to create a new cluster using the standard compute creation form.

To register interest in DataLab safe researcher training, please use the link on the Safe researcher training page.

To request input or output clearance, please use the relevant links on the Input and output clearance page.

Important reminder: do not include unvetted data or counts in emails sent to the ABS

For all other queries, please contact us via email at mydataportal@abs.gov.au. This email account is monitored and we will respond to your query during standard business hours, Monday to Friday.