Troubleshooting

If your DataLab account name is not recognised by your authenticator application, it may be because you have downloaded an authenticator not published by Microsoft. The ABS DataLab only supports Microsoft Authenticator. 

You will need to download the Microsoft Authenticator application to your smart phone from the App Store (for iOS) or Google Play (for Android) to fulfil the login sequence.

The following authenticator applications (similar in appearance to the Microsoft Authenticator) will not function with the ABS DataLab:

If you have verified you are utilising the correct application but are not receiving prompts to authenticate on your mobile device, check your battery optimisation is turned OFF by following these steps. 

1. Open Microsoft Authenticator, and refer to the top right-hand corner, if there is a red dot above the three white dots, press this and then select 'Allow' to turn the battery optimisation off.
2. Then try logging in again on the DataLab landing page.

If you receive the ‘Error communicating with server’ message on your mobile device:

• If your device is connected to the home WiFi, then please turn your phone over to mobile data and try logging in again.

If you are switching to a new phone/tablet device: 

• the operating system of some mobile devices may not interact as expected with Microsoft Authenticator (namely the iOS 16, and some older versions of Android), in this case, try setup another phone or tablet device. In the case that you still have your old phone it may help if the old account is removed from the Microsoft Authenticator application on the previous device.

Other troubleshooting steps you can follow if you are having issues authenticating are:

• Ensuring your phone is connected to the internet (if required switch from WiFi to mobile data to ensure a good connection)
• From your phone's settings menu, ensure push notification are enabled
• Restarting your phone
• Ensuring you have the latest Microsoft Authenticator app version installed
• Clear your phones cache. On iPhone you will need to delete and reinstall the app. For Android navigate to the App storage settings, select 'Storage' and tap 'Clear cache'.
• Ensure your phone is on the latest OS version

1. Your account may have been disabled. Accounts inactive for 45 days are automatically suspended to maintain security. Contact mydataportal@abs.gov.au by submitting a system support query for assistance.
2. If you have entered your user name or password using copy and paste, you may have accidentally included hidden characters or a space.
3. Your organisation firewall may be blocking access. Try accessing DataLab while disconnected from your organisation's network.
4. The ABS DataLab only supports use of the Microsoft Authenticator app.
5. If you have changed your mobile phone we need to reset your Microsoft Multi Factor Authentication, contact mydataportal@abs.gov.au by submitting a system support query.
6. If you need to reset your password this must be done via the 'Forgotten my password' link in the initial DataLab sign in screen. For instructions on how to reset your password refer to Logging into the portal and workspace. 
7. Clear your browser cache.
8. Try a different browser. See Recommended browsers.

DataLab is enabled by cloud infrastructure, which may be blocked by some organisations’ firewall settings.

ABS cannot make changes to external organisations' infrastructure. Project Leads need to supply the information below to each organisation participating on this project.

Network/IT Security sections in each organisation need to review and make changes to authenticate access.

1. Enable authentication to the tenant

Users need to authenticate to one of ABS Azure Active tenants, which may be strictly controlled by government agencies and academic workplaces. Authentication must be enabled to the tenants:

• mydata.abs.gov.au
• absmydata.onmicrosoft.com

2. Allow user access to URLs

Users will need to access the following URLs:

• DataLab production portal: datalab.abs.gov.au and sead.abs.gov.au 

3. Configure your organisation's network to allow outbound connections to the following addresses required for Azure Virtual Desktop (AVD):

• login.microsoftonline.com
• *.wvd.microsoft.com
• *.servicebus.windows.net
• go.microsoft.com
• aka.ms
• learn.microsoft.com
• privacy.microsoft.com
• query.prod.cms.rt.microsoft.com 

These addresses all utilise the TCP protocol and outbound port 443 for communication. Contact mydataportal@abs.gov.au by submitting a system support query for further assistance. 

The DataLab has more functionality and features available to you, so you can set options as well as undertake your research.

• First log-in is to the DataLab portal, where you can view and set options for your DataLab account information and virtual machines. Read more in DataLab portal features.
• Second log-in is to the DataLab workspace where you undertake your analysis.

• The temporary password issued to you by the ABS lasts for 90 days. After you have completed the set up steps you must reset your password.
• If you have forgotten your temporary password, contact mydataportal@abs.gov.au by submitting a system support query for a password reset.

Your login credentials for the DataLab portal are the same as for the DataLab workspace. You can reset your password by clicking on the 'Forgotten my password' link. For instructions on how to reset your password refer to Logging into the portal and workspace. 

If you are locked out of your active session for any reason, including a password expiry you may attempt to reset your password. Active VM sessions may continue in the background until either the nightly shutdown or 14 day rebuild occur.

1. Log into the DataLab portal, Activate your VM, then start the VM. Follow the process and wait for each step to complete before progressing.
2. Check your internet connection. If you have a weak or intermittent connection, this can affect launching your virtual machine.
3. Try launching the virtual machine outside of your organisation's online environment. Some institutions’ or Government departments’ firewall or other security settings may be preventing access to DataLab portal and/or launching of the VM. Attempting to connect outside your agency’s online environment may assist in forming the VM connection.
4. Restart your virtual machine. As with restarting a computer, restarting your virtual machine can sometimes resolve problems with launching your machine successfully. As outlined in the image below, from the virtual machine page click the 'Restart VM button' and wait 10 minutes to ensure the reboot of the machine is complete before attempting to launch again.
5. If you are still having trouble, contact mydataportal@abs.gov.au by submitting a system support query.

If you are a member of multiple projects in the DataLab, you will have more than one virtual machine. Your Active machine is the one that is connected to the remote file share, where the data files are stored. For security purposes, only one of your sessions can connect to the remote file share at a time (this is where data files are stored). You can activate your virtual machine by using the Change Active VM button.

If you receive the error message ‘We couldn’t connect because there are currently no available resources’ or similar when attempting to launch your VM from the Windows App, check that you are using the latest VM version, the machine you are trying to launch is ‘activated’, and that your machine is not currently being rebuilt. These details can be checked in the DataLab web portal.

If you receive this message when returning to a VM after a processing job, you may need to manually restart your VM from the web portal. It is possible for VMs to crash after memory exhaustion due to intensive R processing. Check your code for any issues that may cause memory exhaustion or request for a VM size with larger specifications.

To use multiple displays or adjust the screen size when using DataLab on the Remote Desktop client (MSI) or the Windows App, you will need to adjust the display settings.

You can modify the display settings for your VM in the Remote Desktop Client or Windows App. Open the settings for your VM, open the display settings and turn ‘use default settings’ off. This will show the available display settings. You can then adjust the display configuration to enable multiple displays.

To adjust the screen size, make sure ‘Fit session to window’ is enabled.

Microsoft does not currently offer an AVD client for Linux users. Because of this, we are unable to provide support for Linux users looking to access DataLab. While other options for connecting to the DataLab using Linux may exist, for example setting up a Windows VM on your Linux computer, then connecting to your DataLab VM using the Windows App or Remote Desktop Client, we cannot guarantee its reliability.

Virtual machines are destroyed approximately every 14 days for security purposes. If the 14 day timing will interfere with the timing of your project, you can choose to destroy and rebuild earlier than 14 days at a time that suits you.

Virtual machine Project and Output drives are backed up every night and kept for 14 days. Files outside of these drives are not recoverable.

Save your work to your Project or Output drives to ensure that your analysis is not lost. Information saved outside of these drives is destroyed when your machine is rebuilt every 14 days.

Only if you have requested local disk space to be allocated to a machine. This allows you to run jobs on offline VMs.

Products you have access to can be accessed from the ‘My data products’ folder. If you cannot see this, try logging out and stopping your VM, then begin the Start VM process again. If that does not work, try the rebuild now from your VM management options. If you have just rebuilt your VM, allow additional time for the VM to initiate fully, this may take 10-15 minutes.

When accessing the DataLab through your organisation's virtual machine it is expected that the access would be slow given the multiple levels of virtualisation and processing of inputs. We recommend accessing the DataLab on your base machine, which may make your VM more responsive.

System performance issues can occur for many different reasons, and every issue requires a unique approach to troubleshoot. Many issues can be resolved by attempting one of the following:

• Ensure you have a good and stable internet connection
• Close and reopen the program you are using, and close any other programs or processes that may be consuming system memory
• Confirm your project drive has available space, if not attempt to free up space
• Refresh your network drives using the icon on the workspace desktop
• Attempt to shutdown or rebuild your virtual machine

If issues persist, contact mydataportal@abs.gov.au by submitting a system support query for further assistance. 

If you cannot see the Library, Project, and Output network drives in File Explorer, go to the desktop and double-click the 'Refresh Network Drives' icon. If you have just rebuilt your VM, allow additional time for the VM to initiate fully, this may take 5-10 minutes.

If after waiting, the drive(s) are still not appearing, try stopping and restarting your VM. If they are still not visible, and the refresh network drive button is not appearing on your desktop, you may need to try another VM rebuild.

An error like the following Stata example, may indicate exceeding the available memory allocated to your virtual machine:

Some analytical processes, software tools and methods for working with large datasets are more memory-intensive than others, to troubleshoot attempt the following:

• Use an alternative method/program to manipulate or process the dataset. 
• Most statistical software tools are able to filter data as it is imported. If your analysis only needs variables a, b and c from a dataset containing 30 variables, then selecting, filtering or importing only these variables uses less memory.
• If you cannot do this in your software, consider creating a subsetted data file using another tool, such as Python, as the first step of preparing your data for analysis.
• If you are unsure of alternative methods, we recommend discussing with other researchers in your project team who are more familiar with your chosen statistical software. The ABS does not provide advice or training on using the analytical tools provided to you in the DataLab.
• To request a larger machine or have local disk attached, please submit a change request in the myDATA portal. Larger machines incur higher running costs. With user charging, you may need to consult with your organisation to confirm incurring additional expenses for your project before applying for a larger machine.

If the issue persists, please submit a system support including detailed steps to reproduce the issue in order for ABS support staff to best facilitate a solution.

Users with SAS installed now have the ability to edit the SAS configuration file located at “C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg”. This file governs various software settings and parameters, enhancing customisation options. Noting any changes will need to be repeated following machine rebuilds, as the C drive is destroyed on rebuild. To modify the SAS config file, follow these steps: 

1.    Access the file at the provided directory, using SAS or a text editor. 
2.    Proceed to make the necessary changes, in order to tailor the configuration.
3.    Save the file (and a backup version on your P drive, to reinstate following machine rebuilds), then test the software for desired outcomes. Exercise caution, as improper edits may lead to unexpected behaviour. 

DataLab users can access a pre-written SAS configuration script that enhances SAS efficiency within the DataLab. This includes the "VM_COMPLETE_SETUP.bat" script and supporting resources to streamline the setup process following machine rebuilds. Please refer to the resources located in the Shared Library Drive (L: Drive).

Clean up the drive contents, review and delete redundant files to free up space.

To request a storage increase, contact mydataportal@abs.gov.au by submitting a system support query. There may be a cost associated with this.

You can request transfer clearance for data, code or files to be loaded to your project, from either another DataLab project, or other sources that you hold.

If the code is externally sourced or from a third‑party, a software request should be submitted.

Yes, each virtual machine has R, STATA and Python as default software. SAS is not automatically provided on all machines but can be requested as it requires a licence to be assigned to your virtual machine. To request SAS, please submit a change request in the myDATA portal. 

Jupyter is no longer natively available in DataLab. To install the full Jupyter meta package, enter the following command into Powershell or the Command prompt:

• python -m pip install jupyter

When launching Jupyter Notebook or JuypterLab from Powershell or command prompt, the system may look in the incorrect directory resulting in an error message. To overcome this, enter the following command into Powershell or the Command prompt:

1. To launch Jupyter Notebook, enter: 'python -m notebook'
2. To launch JupyterLab, enter: 'python -m jupyterlab'

For more information please refer to ‘Using Python in the ABS DataLab and SEAD’ guide located in the Shared Library.

Python packages and their dependencies are regularly scanned for known vulnerabilities. Any package versions that exceed the vulnerability threshold will not be available in Posit. If this occurs, check if there are alternative versions of the packages available. In most cases other versions of the package will be available to address known vulnerabilities. If you require assistance, please contact mydataportal@abs.gov.au by submitting a system support query.

Yes, it takes about 5 minutes to process the connection. You also need to log out of your virtual machine to allow the system to refresh your session with the new data.

You can extend your session to bypass the nightly shutdown, by up to three nights.

Use the R Studio Package Manager shortcut on the DataLab virtual machine desktop to check the range of R packages available to you. See Managing your R packages.

By default, packages are installed onto the C drive which gets wiped every time the VM rebuilds. To have packages which persist through rebuilds, we recommend changing the file path to have packages installed on either the Project drive (P:) or a local disk (if attached). Files saved to these drives are retained over rebuilds.

If RStudio is running slowly or lagging, one possible cause is background diagnostics. RStudio continuously analyses code for errors and warnings, and on large scripts this can slow things down. Turning off diagnostics can improve performance.

How to disable R diagnostics in RStudio:

1. Go to Tools → Global Options.
2. Select Code from the left panel.
3. Click the Diagnostics tab.
4. Untick Show diagnostics for R.
5. Click Apply, then OK.

This reduces background processing and may make RStudio more responsive.

If the project you opened was saved with SAS Datalab – [machine name] you are connecting to the local SAS server without a profile. When you try to run the project without selecting a profile the system may present an error message saying "The server "SASMain" is not defined in the current repository". Click though the messages and continue.

1. See Managing your R packages to use the RStudio Package Manager on the desktop.
2. If the packages you need is not listed, please contact mydataportal@abs.gov.au by submitting a Request new software or software packages query. 

Due to a default setting in Microsoft, the system automatically uses Microsoft Edge to open any PDF file. If you prefer to use Adobe Acrobat Reader, you can open the file by right-clicking the file, selecting 'Open with' and then selecting 'Adobe Reader'. The file will then launch using Adobe Acrobat Reader.

Following recent enhancements, PostgreSQL's data directory relocation is now automated to X:\psql\data when a local disk is attached, facilitating expanded storage capacity for data. This feature streamlines the process of accommodating larger datasets. To enable local disk, please contact mydataportal@abs.gov.au by submitting a system support query. 

When creating a new compute cluster in Databricks there is a known issue with the new ‘Simple’ compute creation form. This is identified with the red triangle next to the create cluster button. Ensure that the Simple form setting at the top of the create cluster page is switched to ‘Simple form: OFF’ when creating a cluster. This will enable you to create a new cluster using the standard compute creation form.