1504.0 - Methodological News, Dec 2001  
Previous ISSUE Released at 11:30 AM (CANBERRA TIME) 15/11/2001   
   Page tools: Print Print Page Print all pages in this productPrint All RSS Feed RSS Bookmark and Share Search this Product

USEABILITY TESTING OF ELECTRONIC FORMS

The Forms Consultancy Group (FCG) are stakeholders, together with sections in the Technology Services Division and the Economic Statistics Group, in the ABS' response to the Electronic Transactions Act (ETA). The ETA requires the ABS to accommodate businesses and individuals who insist or need to report to us electronically. While other, more effective electronic forms are being developed, MS EXCEL spreadsheets were considered the most cost effective and viable option in the short-term and will be used as the 'ETA-fallback' electronic data collection vehicle for businesses.

The FCG have conducted extensive research into electronic reporting and have an approval role for ABS electronic data collection instruments. We coordinated the testing of the ETA fall-back MS EXCEL forms to ensure the design would be reasonably acceptable to respondents. As the forms were intended to be as cheap and easy to produce as possible, we conducted a useability test in the Research and Design Centre, using ABS staff as our test subjects. With detailed scenarios they were able to pretend to be reporting for a real business quite effectively.

The first round of testing primarily examined navigation, using four versions of the form each with a different method of moving through the worksheets. Each subject filled out one of the versions using their scenario and then completed an evaluation questionnaire. For comparison, the subject was then quickly shown another version and asked to explain any preference. A balanced number of version-pair combinations allowed us to examine which versions were preferred overall and why.

None of the methods emerged as more strongly preferred by users than the others. Some of the navigation results were:

  • labels on spreadsheet tabs, links or buttons that refer to page numbers in the form weren't very useful;
  • the varying positions of the above navigational devices were a confounding factor in assessing their functionality;
  • scrolling through a form that was one long page wasn't anywhere near as irritating to subjects as expected;
  • using macros to solve any of our design problems is out of the question - subjects turned them off when they opened the form almost every time; and
  • the form needed to provide much better keyboard navigation.

Some of the other results from the first round of testing included:
  • subjects had a very strong desire for automatic totals;
  • having a form in the corner of a large spreadsheet caused subjects to get lost in the surrounding white space; and
  • answer boxes that need to take varying lengths of text don't work well in MS EXCEL.

After combining what we hoped were the best navigation elements into a single form, and addressing most of the glaring or easily fixed design problems, we tested again. The new form included spreadsheet tabs labelled with sections of the form, automatic totals and instructions about keyboard navigation. Some of the previous subjects and some fresh ones went through the new form with another scenario to ensure the changes we had made were actually improvements. This was indeed the case. A few more minor improvements and the MS EXCEL 'ETA fallback' electronic form will go live.

For more information, please contact Emma Farrell on (02) 6252 7316.

E-mail: emma.farrell@abs.gov.au