DataLab
Analyse the most detailed microdata in the secure DataLab for your statistical research or modelling, find out about charges and how to access
\(\Large ⚿\) Log into DataLab
What is DataLab
DataLab is the analysis solution for high-end users who want to undertake real time complex analysis of detailed microdata. Compare data services to see if detailed microdata in the DataLab is the right service for you.
Note: Government organisations seeking to leverage the DataLab cloud infrastructure to host their own data and manage their own end-users should consider the Secure Environment for Analysing Data (SEAD) service.
Features
- View and analyse unit record information
- Recent versions of analytical software, including R, SAS, Stata and Python
- Virtual access to files that remain in the secure ABS environment
- All analytical output that you want to use outside DataLab are checked by the ABS before release
Who can access the DataLab
Detailed survey and integrated microdata are available for approved projects, organisational users must be:
- government employees
- government contractors and individuals sponsored by government
- academics
- researchers from public policy research institutes
- sponsored by government
All users need to also meet ABS safe people criteria, including researchers who:
- belong to an Australian organisation (international researchers and organisations will be considered on a case by case basis)
- belong to an organisation with a Responsible Officer Undertaking (ROU) in place with the ABS
- are located in Australia when accessing the microdata
- have completed all relevant undertakings and declarations
- have the ability to use at least one of the statistical analytical languages available in the DataLab
- have at least three years of either quantitative research experience or university study with a significant component working with quantitative data, or have a referral from an experienced researcher working on the same project
- have an approved safe project that is for statistical and/or research purposes and demonstrates public value
- have completed ABS safe researcher training and refresher training as per ABS refresher policy
- meet additional criteria that apply to specific microdata.
DataLab system security
The ABS is committed to keeping the ABS DataLab safe and secure. We have a strong data protection culture and extensive experience in keeping data secure as Australia’s national statistical organisation and as an Accredited Data Service Provider. The ABS DataLab is hosted in Microsoft Azure and meets PROTECTED level security standards as prescribed in the Australian Government Information Security Manual (ISM). It is subject to Independent Security Registered Assessors Program (IRAP) certification, ongoing security audits and robust IT security testing and patching delivering the Safe Settings aspect of the Five Safes Framework.
The technology underpinning the ABS DataLab includes:
- data encryption at rest to mitigate against unauthorised access to microdata
- Azure Storage Accounts to securely hold individual research products and allow querying from authorised users
- cloud servers (including backup servers) hosted exclusively onshore, with access only authorised for use in Australia unless approved by the ABS
- closed network virtual machines to provide secure, isolated research spaces for the analysis of microdata
- guarded access through multi-factor authentication and workspace segmentation inhibiting data sharing between projects
- a DataLab Product Storage Account protected with Microsoft Defender providing threat detection against malicious/unusual behaviour.
The ABS employs the above with a focus on industry standard security posture management to provide a safe and secure platform for policy and program delivery work.
Detailed microdata in the DataLab
- Designed specifically for use within the DataLab environment
- Direct identifiers (such as names and addresses) removed
- Further appropriate confidentiality applied within the context of the other security features of the DataLab
- Topics include Census, health, education, labour force, Aboriginal and Torres Strait Islander peoples, migrants, crime, business, disabilities, ageing and carers
- Datasets include ABS survey results, administrative data collected by other organisations and integrated datasets
- Data item lists are linked in detailed microdata topics in the DataLab
Charges
Costs for 2024-25 are now available below. If you have any questions, please contact data.services@abs.gov.au.
Approved users can access standard detailed microdata in DataLab for approved projects. This includes:
- ABS survey and census collections
- data ABS has collected from other organisations (with custodian approval)
- integrated microdata such as:
- Person Level Integrated Data Asset (PLIDA)
- Business Longitudinal Analysis Data Environment (BLADE) Core plus BLADE standard module (various ABS surveys), Intellectual Property Longitudinal Research Data (IPLORD) and Merchandise Imports and Exports
- PLIDA/BLADE linked data
Additional charges apply for customised data integration services.
DataLab charges
DataLab access incurs an annual charge. This charge is based on the number of analysts with virtual machine access in a project. The charge covers the annual costs of:
- project establishment and ongoing administration and support
- researcher onboarding (including training and refresher training)
- changes to analysts and discussants within a project
- standard virtual machine access for analysts
- standard software access for analysts (e.g., R, Python, Stata)
- project storage up to one terabyte
- standard output and input clearance
The ABS is committed to supporting the DataLab service and subsidising DataLab users. In 2023-24, increasing costs and budget constraints require the ABS to move towards a more sustainable partial cost recovery arrangement with all our clients. The below updated charges will allow the ABS to maintain its service levels as well as deliver critical system and infrastructure enhancements.
The ABS is adopting a staggered approach to price increases to minimise the impacts on existing projects. Existing projects are projects established before 1 July 2023. Existing and new projects will be subject to different pricing models for 2023-24 and 2024-25. Please see the below for more details.
Please note:
- Individual quotes will be prepared for projects with over 25 analysts.
- An increase in the number of analysts to the next tier will incur an additional charge equivalent to the next tier.
- Significant changes in project scope may result in the establishment of a new project.
- Project extensions after 1 July 2023 will be charged at the new project rates.
- Access to non-standard services are subject to additional charges, please see ‘Additional non-standard access and services charges’ table below.
- All charges are calculated quarterly, based on the month of request. For example, a project commencing in October will be charged the annual fee and any non-standard DataLab access based on three quarters of the financial year.
- Goods and Services Tax (GST) will be applied to charges unless the organisation is exempt.
Annual charges for existing projects
Existing projects are projects established before 1 July 2023.
Tier 1 and 2 projects (projects with under 10 analysts)
- Tier 1 and 2 projects will continue to be charged the same annual rates for financial years 2023-24 and 2024-25.
- Prices will increase from 2025-26 for all Tier 1 and 2 projects.
Tier 3 projects (projects with 11 to 25 analysts)
- Tier 3 projects will be charged half the 2022-23 annual charge of $10,000 ($5,000 excluding GST) for 1 July 2023 to 31 December 2023, with prices increasing from 1 January 2024.
- From 1 January 2024, Tier 3 projects will be subject to a new charge rate.
- Tier 3 projects will be charged half the new 2023-24 annual charge of $30,000 ($15,000 excluding GST) for 1 January 2024 to 30 June 2024.
- The delayed roll out of the new charging model provides project leads the opportunity to decrease usage or close their project before prices increase.
Tier 4 projects (projects with over 25 analysts)
- Tier 4 projects will be charged custom pricing from 1 July 2023 under the new pricing model.
| Annual charge (per project) | Excluding GST |
|---|---|
| Tier 1 - 1 to 5 analysts | $2,000 |
| Tier 2 - 6 to 10 analysts | $4,000 |
| Tier 3 - 11 to 25 analysts | $5,000 - 1 Jul 2023 to 31 Dec 2023 $15,000 - 1 Jan 2024 to 30 Jun 2024 |
| Tier 4 - Over 25 analysts | Custom |
Annual charges for new projects
New projects are projects established after 1 July 2023.
| Annual charge (per project) | Excluding GST |
|---|---|
| Tier 1 - 1 to 5 analysts | $5,000 |
| Tier 2 - 6 to 10 analysts | $12,000 |
| Tier 3 - 11 to 25 analysts | $30,000 |
| Tier 4 - Over 25 analysts | Custom |
University projects
For projects and data services covered by the current ABS/Universities Australia Agreement, students, staff and researchers are not individually invoiced for their use of the charged ABS data services.
For more information refer to the ABS/Universities Australia Agreement.
New projects may progress their access requests outside of the ABS/Universities Australia Agreement, and will be subject to access fees and charges as outlined on the DataLab page.
ABS International Merchandise Trade (IMT) statistics charges
ABS International Merchandise Trade (IMT) statistics are subject to additional disclosure controls and consistency checks with published ABS data when releasing IMT data from the DataLab. These additional activities incur additional costs. From 2024-25 projects requiring access to IMT data are subject to the following charges:
- $500 (GST exclusive) – per project requesting access to merchandise imports or exports data.
- $250 (GST exclusive) – per IMT output request submitted.
Additional non-standard access and service charges
Non-standard charges apply to all projects for services that are not within the scope of the annual charge. Non-standard charges include access to SAS, Databricks, non-standard virtual machines and storage above one terabyte. Non-standard charges also apply to increased service levels, such as priority clearance and high service level. More information on non-standard access and services are in the tables below.
| Annual charges per person per project | Excluding GST |
|---|---|
| Use of SAS | $500 |
Non-standard virtual machines Standard virtual machines are included in the annual charge and comprise machines up to and including the large size. Please refer to virtual machines for further information on size. Access to non-standard virtual machines is subject to a minimum charge. Should analysts exceed their usage in dollar terms for their non-standard virtual machine within the financial year, access can continue subject to additional charges being applied. Usage is calculated per hour. Please see virtual machines for hourly rates. If shorter term usage of a non-standard virtual machine is required, please contact data.services@abs.gov.au to discuss options. | $1,700 (minimum) |
| Annual charges per project | Excluding GST |
|---|---|
Databricks - low usage Databricks - high usage Please refer to Databricks for more information on this service. The price of this access is a minimum charge. Should analysts exceed their usage in dollar terms for their access to Databricks within the financial year, access can continue subject to additional charges being applied. | $3,500 (low usage minimum) $6,500 (high usage minimum) |
Each additional terabyte of storage One terabyte of storage is included in the annual charge. | $850 |
| Annual charges per project | Excluding GST |
|---|---|
High output demand - Tier 1 High output demand - Tier 2 A high output demand charge applies to projects that require a higher level of service, including high volumes, faster turnaround or the application of special rules. This charge will be applied when the project team requests this higher level of service, or when the ABS determines that a project is requiring resources exceeding cost recovery of the annual charge. High output demand charges are structured into two separate tiers. Tier 1 applies to projects with output requests that:
Tier 2 applies to projects with output requests that:
| $19,000 (Tier 1) $41,000 (Tier 2) |
High service level A high service level charge applies to projects that require resources exceeding the cost recovery of the annual fee. This charge will be applied when the project team requests this higher level of service, or when the ABS determines that a project is requiring resources exceeding cost recovery of the annual charge. The following will be a factor in considering the application of the high service level charge:
| $20,000 |
Location Modular Product access charges
Charges apply to all projects accessing the Location Modular Product. More information on Location Modular Product services and associated charges are in the tables below.
| Annual charges per project | Excluding GST |
|---|---|
| Data Integration Project assessment charge | |
| Simple The simple assessment charge will apply, in most instances, to projects with resupply or reuse of existing data arrangements. | $4,577 |
| Complex The complex assessment charge will apply, in most instances, to projects with new data integration requests or refreshes of existing datasets. | $6,774 |
| Privacy charges | |
Data Integration Plan
| $6,038 |
| Privacy Threshold Assessment (PTA) – Simple A PTA is a preliminary assessment that helps to determine a project's potential privacy impacts. This includes consideration of whether a project is high risk and should be subject to a PIA as required by the Australian Government Agencies Privacy Code. New projects that propose to integrate one dataset will require a simple PTA. Existing projects proposing to integrate an additional dataset will also require a simple PTA. | $ 3,071 |
| Privacy Threshold Assessment (PTA) - Complex A PTA is a preliminary assessment that helps to determine a project's potential privacy impacts. This includes consideration of whether a project is high risk and should be subject to a PIA as required by the Australian Government Agencies Privacy Code. New projects that propose to integrate multiple datasets will require a complex PTA. Existing projects proposing to integrate multiple additional datasets will also require a complex PTA. | $8,867 |
| Privacy Impact Assessment (PIA) A PIA is a systematic assessment that helps the ABS identify and manage the privacy impacts of a data integration project. Custom charges apply as the size, scale and scope of PIAs vary greatly depending on the need for stakeholder consultation, the nature of the data, the risk level of the project and the complexity of changes to personal information and data handling practices. PIAs may be conducted by the ABS or by independent consultants. | Custom |
| Other privacy measures Additional privacy measures are required for all new types or categories of data proposed for integration with PLIDA. This includes targeted stakeholder consultation and additional transparency activities. | Custom |
| Data Supply Agreement charges | |
| Simple A simple data supply agreement is generally a straightforward Letter of Exchange (as opposed to an MoU or more complex agreement) with a single data custodian, a small number of datasets and linkage to either PLIDA or BLADE (not both). | $5,221 |
| Complex A complex data supply agreement is generally a more detailed document often covering a larger number of datasets to be supplied, multiple data custodians and may have linkage requirements for both PLIDA and BLADE. | $6,808 |
| Data Integration (linkage and assembly) charges per dataset | |
| Geocoding/Hashing address information Charge for Data Linkage Centre to geocode or hash address information. | $3,021 |
| Basic Data set contains Address information only. Address information provided is clean and good quality. Dataset only used to scope PLIDA or BLADE to population of interest. | $5,750 |
| Small dataset A small dataset is comprised less than and no more than 30 variables in total. Address information provided is clean and good quality. The dataset also contains no additional identifiers that would require de-identification and does not require non-standard confidentiality treatments. | $10,477 |
| Medium dataset A medium dataset is comprised no more than 100 variables in total. Alternatively, a medium dataset may be smaller than this but contain additional identifiers that require de-identification, or requires other non-standard confidentiality treatments. | $16,721 |
| Large dataset A large dataset is comprised of more than 100 variables. | $28,313 |
| Data resupply In the event that data needs to be re-supplied due to major corrections required at the data provider end, additional fees may apply to cover the costs of additional checks to ensure data is suitable for integration. | $2,404 |
Applying for DataLab access
Step 1. Ensure you meet requirements
For criteria, refer to Who can access the DataLab
Organisation approval
- Your organisation must have a verified Responsible Officer Undertaking (ROU) in place with the ABS. If one does not exist, your organisation will not be available for selection in the myDATA online project proposal.
- To check if there is an active ROU for your organisation, go to the myDATA user portal homepage and select Dashboard/Organisation.
Researcher approval
- You must have a commitment to protect the confidentiality of data.
- Every member of your project team who will see or discuss uncleared outputs (whether or not they will be using DataLab) needs to be approved.
Project approval
- Projects must be for statistical and/or research purposes and provide public benefit.
- Projects must not be for compliance or regulatory purposes.
- Every project needs to be approved by the ABS.
- Projects for or about Aboriginal and/or Torres Strait Islander peoples may be subject to a Cultural Review by the Centre of Aboriginal and Torres Strait Islander Statistics at the ABS.
- Some projects also require consideration and approval by data custodians.
Refer to What is DataLab and Using DataLab responsibly for more information.
Step 2. Register and activate your account
Registration of an account will allow you to:
- create a project proposal within the myDATA user portal
- enrol in DataLab safe researcher training
- collect forms for your onboarding process after training has been completed
- draft and review projects you are participating in.
Register in the myDATA Portal and agree to the Conditions of use
- Use your organisation email address. If you are a user in more than one organisation, you will need to register separately using the email address for each organisation.
- Authenticate your account - myDATA will automatically email your registered account with steps to authenticate.
- If you encounter errors in myDATA, please submit a System support query.
You can complete your training while your project proposal is underway.
Access will not be granted until DataLab safe onboarding is completed. For the onboarding process, the following documents are required to be submitted:
- Individual Undertaking and DataLab Declaration of Compliance
- DataLab quiz
- additional documents relevant to particular datasets, if requested by ABS
- if you are not an experienced researcher, you may need to submit a DataLab Referral, please email data.services@abs.gov.au to request this form.
For further information, please refer to the myDATA user guide.
Step 3. Submit project proposal
| Create and complete a new project proposal in the myDATA user portal | Updates to an existing project proposal (word document) |
|---|---|
The creator of a new project will be automatically assigned as the Project Editor, they are the only person who can edit and submit the project proposal. Project Leads can be identified separately when adding people to the project. Step through the online form and complete all required fields. Submit your completed project proposal to the ABS for review, who will respond with feedback if edits are required. Note: researchers must complete registration and activation steps successfully before they can be added to the project. Training does not have to be completed to be added to a project – access will not be granted until after approval is given. For further information, please refer to the myDATA user guide. | For existing projects, the project proposal must be updated with any changes (e.g., changes to researchers, organisations, data or scope) and submitted to data.services@abs.gov.au Changes to project proposals must be made in red, with tracked changes on, and supplied on the newest version of the template below. Project proposal template and data request form |
Enabling access to DataLab
DataLab is enabled by cloud infrastructure, which may be blocked by some organisations’ firewall settings.
ABS cannot make changes to external organisations' infrastructure. Project Leads need to supply the information below to each organisation participating on this project.
Network/IT Security sections in each organisation need to review and make changes to authenticate access. This only needs to be done once per organisation.
Azure Virtual Desktop configuration
1. Enable authentication to the tenant
Users need to authenticate to one of ABS Azure Active tenants, which may be strictly controlled by government agencies and academic workplaces. Authentication must be enabled to the tenant:
- absmydata.onmicrosoft.com
This tenant is in the Azure Australia East and Azure Australia Central regions.
2. Allow user access to URLs
Users will need to access the following URLs:
- DataLab production portal: datalab.abs.gov.au and sead.abs.gov.au
3. Configure your organisation's network to allow outbound connections to the following addresses required for Azure Virtual Desktop (AVD):
- login.microsoftonline.com
- *.wvd.microsoft.com
- *.servicebus.windows.net
- go.microsoft.com
- aka.ms
- learn.microsoft.com
- privacy.microsoft.com
- query.prod.cms.rt.microsoft.com
These addresses all utilise the TCP protocol and outbound port 443 for communication.
Contact data.services@abs.gov.au for further assistance.
Accessibility and inclusiveness
The ABS is actively working to comply with Web Content Accessibility Guidelines (WCAG) 2.1 at Level AA, which describe how to make web content more usable and accessible for everyone. Our goal is to provide information that is accessible to all, including people with disabilities.
Recent accessibility reviews confirm that the DataLab/SEAD cloud platform meets the necessary organisational standards as it:
- runs on Windows 11, which enables the utilisation of existing accessibility features and functions
- utilises virtual machines that are subject to Microsoft compliance obligations
- contains statistical tools that are either WCAG compliant or provide integration with Windows Accessibility features
Note: Any Windows accessibility features not enabled by default can be requested, and will be considered in line with relevant data sensitivities.
Privacy policy
The ABS privacy policy and DataLab privacy notice outline how the ABS handles any personal information that you provide to us.