4363.0.55.001 - Australian Health Survey: Users' Guide, 2011-13  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 09/05/2014   
   Page tools: Print Print Page Print all pages in this productPrint All  
Contents >> Nutrition >> Interpretation of Results >> Under-reporting in Nutrition Surveys

UNDER-REPORTING IN NUTRITION SURVEYS

There are a range of possible sources of error related to survey data. Of particular importance to nutrition surveys is a widely observed tendency for people to underestimate their food intakes1. This is called under-reporting and can include:

  • actual changes in foods eaten because people know they will be participating in the survey
  • misrepresentation (deliberate, unconscious or accidental), e.g. to make their diets appear more ‘healthy’ or be quicker to report.

The Automated Multiple-Pass Method (AMPM) used in the 2011-12 National Nutrition and Physical Activity Survey (NNPAS) developed by the United States Department of Agriculture (USDA) uses several methods to assist respondents to recall their food and beverage intake in order to reduce the amount of under-reporting in nutrition surveys. Even so, an evaluation of the performance of this instrument against an independent measure of intake (i.e. energy expenditure measured by doubly-labelled water) in 524 volunteers from the Washington DC area showed that although the AMPM accurately reported energy intakes for normal weight subjects, under-reporting remained an issue for overweight and obese subjects2.

Analysis of the results of the 1995 National Nutrition Survey (NNS) and the 2011-12 NNPAS suggests that, like other nutrition surveys, there has been some under-reporting of food intake by participants in these surveys and that the patterns of under-reporting have changed over time. In making comparisons between the two time points, it is important to also consider that:
  • population mean body weights have increased in Australia since 19953
  • energy requirements (and intakes) increase with increasing body weight and increasing physical activity levels4
  • barring a substantial reduction in physical activity amongst Australians since 1995, the fundamental principles of energy physiology suggest that true population energy intakes are unlikely to have substantially decreased.5

A common approach used to identify under-reporters in surveys is to compare each person's basal metabolic rate (BMR) (based on their age, sex and weight) with their reported energy intake (EI) and apply Goldberg cut-off values to assess whether the energy intake reported is plausible. BMR represents the amount of energy expended at rest over a 24-hour period by an individual (see Nutrient Intakes for method of calculation). The EI:BMR ratio provides an indication of whether the reported energy intake for one day is consistent with the energy intake required for a person to live a normal (not bed-bound) lifestyle. The habitual energy expenditure by an individual will exceed their BMR, mainly as a result of physical activity. It is therefore expected that habitual energy intake will be greater than BMR. A lower than expected EI:BMR value may indicate dieting, unusually low consumption or under-reporting of food consumption during the 24-hour reference period.5

Goldberg et al. developed the use of the EI:BMR ratio as a method of establishing cut-off limits for determining those adults whose reported energy intakes were incompatible with the amount of energy required for a normally active but sedentary population (not sick, disabled or frail elderly). This is equivalent to a physical activity level or EI:BMR ratio of 1.55. Using this value, Goldberg et al. established a cut-off limit of 0.9 for EI:BMR for a plausible intake, which is the lower 95% confidence limit for a single day of data for a single individual, allowing for day-to-day variation in energy intakes, and errors in calculation of EI:BMR.5

Low Energy Reporters

The prevalence of Low Energy Reporters (LER) as measured by the Goldberg cut-off (EI:BMR<0.9) for a sedentary population for one day’s intake on an individual basis was published for the 1995 NNS6. Applying the same method to the 2011-12 NNPAS data shows substantial increases overall in the proportion of LERs, particularly for males. Similar to the approach taken in the 1995 NNS, results are given for respondents aged 10 years and over only, as the Goldberg cut-off is not useful for growing children7.


Low Energy Reporters in 1995 NNS and 2011-12 NNPAS as Classified by the Goldberg Cut-Off of 0.9: Comparison by Age and Sex (a)


Males
Females

Age
%
%
Percentage points
%
%
Percentage points

1995
2011-12
Difference
1995
2011-12
Difference

10-13
4
10
6
7
18
10
14-18
9
22
12
17
20
3
19-30
8
16
8
17
26
8
31-50
11
20
9
19
23
5
51-70
14
22
7
23
24
1
71 and over
15
18
4
22
21
-1
All 10 and over
11
19
8
19
23
4

(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR. All proportions are given as a weighted proportion of respondents for whom EI:BMR could be calculated.


Low Energy Reporters in 1995 NNS and 2011-12 NNPAS as Classified by the Goldberg Cut-Off of 0.9: Comparison by BMI and Sex (a)


Males
Females

BMI
%
%
Percentage points
%
%
Percentage points

1995
2011-12
Difference
1995
2011-12
Difference

Normal
5
10
5
12
16
4
Overweight
11
18
7
22
24
1
Obese
23
34
11
35
37
2

(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR. All proportions are given as a weighted proportion of respondents for whom EI:BMR could be calculated.


Overall the proportion of people recording an implausible intake was higher for females and increased with BMI. The age pattern varied for males with the youngest age group (10 -13) having the lowest proportion of LERs followed by males aged 19-30 and 71 years and over.

While females were still more likely, in 2011-12, to be LERs than males, there was a greater increase since 1995 for males than females (8 percentage points compared to 4 percentage points). The greatest increase was for males aged 14 – 18 years (12 percentage points).

Obese people were more likely to be LERs than either normal or overweight people at around one in three obese people being LERs for both males and females in 2011-12. This rate has remained constant for females since 1995 (35% in 1995 compared with 37% in 2011-12), however has increased by 11 percentage points since 1995 for males (from 23% in 1995 to 34% in 2011-12). There have also been increases for males in the normal and overweight categories, however these have been smaller.

An increase in the proportion of LERs in males has also been observed in the 2008-09 New Zealand Adult Nutrition Survey compared to the previous cycle conducted in 1997 where the proportion of LERs more than doubled among men classified as normal weight (6.1 to 14.7%)8.

Considering the proportion of LERs in males has increased and is now more similar to females, the increase in under-reporting in males could be from an increased influence of psychosocial factors such as social desirability and eating restraint for males as reported in other studies9.

The Goldberg cut-off, when used in this way on a single day of intake and without overall energy expenditure from physical activity, has been estimated to find only about half of all actual under-reporters10. That is, after making the conservative assumption that the person is sedentary, and allowing for day-to-day variation in intakes and errors in calculating BMR, the cut-off only classifies someone as an under-reporter when their reported intake is so low that it is implausible (with 95% confidence) for their age, sex and weight11. Therefore, removal of LERs from the dataset does not remove all under-reporting bias12. Further, a substantial decline in energy intakes for males (8%) from the 1995 NNS to the 2011-12 NNPAS (12,100kJ compared to 11,080kJ) remains between plausible energy reporters from both surveys (data not presented).

Energy Intakes and Under-Reporting Compared with 1995 NNS

A comparison of energy intakes between 1995 and 2011-12 is another way of analysing the extent of change in the levels of under-reporting.

Reported daily mean energy intakes in the 2011-12 NNPAS declined by 9% in comparison with the 1995 National Nutrition Survey (NNS), with males’ energy intakes down by 12% and females’ by 4%. The largest reductions have occurred in males aged 9 to 50 years, particularly adolescent males (14-18 years) whose reported intakes are down by 22% or 2,797 kilojoules (kJ).

The following graphs show how the whole distribution of reported energy intakes has decreased between 1995 and 2011-12 for males while remaining relatively consistent for females. For example while 47% of males reported energy intakes of less than 10,000kJ in 1995, this percentage increased to 60% in 2011-12. At the same time, 60% of females reported energy intakes of less than 8,000kJ in 1995, this percentage remained relatively constant at 63% in 2011-12.


Graph Image: Males 2 years and over: energy intake, 1995 NNS to 2011-12 NNPASGraph Image: Females 2 years and over: Energy intake, 1995 NNS to 2011-12 NNPAS

One explanation for the change (which cannot be excluded as physical activity data is not available from the 1995 NNS) is that physical activity levels in adolescent and young men have substantially decreased and/or sedentary behaviour has increased, and the drop in reported energy intakes reflects a true decrease. In this scenario the reduction in energy intakes has not been sufficient to fully compensate for the drop in energy expenditure as the proportion of overweight/obese males has increased (up from 63.8% in 1995 to 70.3% in 2011-12)3 over the intervening period. However, a very substantial drop in physical activity, particularly amongst adolescent and young adult males, would be required to solely explain the lower reported energy intakes in 2011-12 than 1995.

The following table shows the mean energy intake by age and sex for the 1995 NNS and the 2011-12 NNPAS, as well as the EI:BMR ratio.

Mean Energy Intake and EI:BMR Ratio in 1995 NNS and 2011-12 NNPAS: Comparison by Sex and Age


Males
Females

Age
1995
2011-12
Difference
% Change
1995
2011-12
Difference
% Change

Mean Energy Intake (kJ) (a)

10-13
10,992
9,391
-1,601
-15
8,596
7,928
-668
-8
14-18
12,983
10,186
-2,797
-22
8,792
8,114
-677
-8
19-30
13,168
11,004
-2,164
-16
8,523
7,863
-660
-8
31-50
11,449
10,220
-1,229
-11
7,829
7,540
-289
-4
51-70
9,773
9,345
-429
-4
7,016
7,268
252
4
71 and over
8,429
8,174
-255
-3
6,432
6,570
138
2
10 and over
11,384
9,934
-1,450
-13
7,795
7,497
-298
-4
Total 2 years and over
10,975
9,655
-1,320
-12
7,727
7,402
-325
-4
Mean EI:BMR ratio (kJ) (b)

10-13
1.84
1.52
-0.31
-17
1.57
1.44
-0.14
-9
14-18
1.69
1.34
-0.35
-21
1.44
1.31
-0.13
-9
19-30
1.69
1.41
-0.28
-16
1.43
1.31
-0.12
-8
31-50
1.51
1.32
-0.19
-12
1.34
1.29
-0.05
-4
51-70
1.39
1.30
-0.09
-6
1.24
1.29
0.04
3
71 and over
1.37
1.31
-0.06
-4
1.25
1.27
0.02
2
10 and over
1.55
1.35
-0.20
-13
1.35
1.30
-0.05
-4

(a) All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.
(b) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.

Mean energy intakes for males aged 10 years and over were significantly lower (-1450kJ, 13%) in the 2011-12 NNPAS than in the 1995 NNS, and the decrease in mean EI:BMR ratio (1.55 in the 1995 NNS compared with 1.35 in the 2011-12 NNPAS) is statistically significant. The largest reductions have occurred in males aged 10-50 years of age. Taken at face value, the reported energy intakes for males are only enough to support the requirements of being ‘sedentary’ to ‘performing light activity’ in 1995 and being between ‘bed rest’ and ‘very sedentary’ in 2011-1213.

While also significantly lower, the decrease for females in mean energy intake was not as great with a decrease of 298kJ or 4% for females aged 10 years and over in 2011-12 compared to 1995. The decrease in mean EI:BMR ratio, although not as great (1.35 in 1995 NNS compared with 1.30 in 2011-12 NNPAS), is significant overall. Taken at face value, the reported energy intakes for females are only enough to support the requirements of being between ‘at bed rest’ and ‘very sedentary’ at both time points13.

For both males and females, the change in EI:BMR is significant in every particular age group (although it is only barely significant in some age groups for females) with the exception of males and females aged 71 years and over. This suggests that, assuming that males and females were about as physically active in 1995 as in 2011-12, the level of under-reporting by female respondents overall has increased in all age groups with the exception of those aged 71 years and over, but not to the same extent as it has for males.

The reduction in reported energy intakes since 1995 for males has been concentrated in higher carbohydrate and fat food groups, in a pattern consistent with an increase in under-reporting, where respondents report what is perceived to be a more desirable dietary intake1.

Factors associated with low energy reporting

The most consistent characteristics associated with low energy reporting in past studies are that females under-report more than males, and obese/overweight respondents report less food/energy than those of normal weight status, when they in fact consume more food/energy. Other factors investigated in limited studies include the inconvenience of spending significant time responding to the survey and social desirability1.

A multivariate regression analysis was conducted on pooled 1995 NNS and 2011-12 NNPAS data to investigate the factors that were the most significant drivers for a person reporting a lower energy intake than their predicted requirement for their age, sex and body weight (i.e. EI:BMR).

This analysis showed that the decrease in EI:BMR ratio between surveys was not due solely to an increase in men at higher BMIs (who tend to under-report more), although this has contributed to it. Many other covariates including dietary restriction, perception of health as poor, perception of self as overweight, and weekend vs weekday intake were significant in the model, but each only explained a very small additional amount of the variation (less than 1%), and collectively they could not explain the difference between the surveys for men. The model was only able to explain approximately 15% of the variation in EI:BMR, although this is not unexpected given the highly variable nature of day-to-day energy intakes for individuals.

The following graphs show that lower energy intakes were reported by normal weight, overweight and obese males in the 2011-12 NNPAS than their counterparts in the 1995 NNS. Energy intakes for females have remained relatively consistent by BMI category over the same time period.


Energy Intake for males and females by BMI category: 1995 NNS and 2011-12 NNPAS (a)

Graph Image: Males, Normal BMI: Energy IntakeGraph Image: Females, Normal BMI: Energy Intake
Graph Image: Males, Overweight BMI: Energy IntakeGraph Image: Females, Overweight BMI: Energy Intake
Graph Image: Males, Obese BMI: Energy IntakeGraph Image: Females, Obese BMI: Energy Intake

(a) Energy intakes presented are for all respondents aged 10 years and over. All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.


Mean Energy Intake and EI:BMR Ratio in 1995 NNS and 2011-12 NNPAS: Comparison by Sex and BMI


Males
Females

BMI
1995
2011-12
Difference
% Change
1995
2011-12
Difference
% Change

Mean Energy Intake (kJ) (a)

Normal weight
12,104
10,601
-1,503
-12
8,114
7,873
-240
-3
Overweight
11,068
9,887
-1,181
-11
7,366
7,548
182
2
Obese
10,596
9,441
-1,155
-11
7,200
7,165
-35
0
Mean EI:BMR ratio (kJ) (b)

Normal weight
1.75
1.54
-0.21
-12
1.47
1.42
-0.04
-3
Overweight
1.47
1.30
-0.16
-11
1.25
1.28
0.03
2
Obese
1.26
1.12
-0.14
-11
1.09
1.08
-0.02
-2

(a) All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.
(b) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.


Earlier in the paper it was shown that the proportion of LERs had increased at different rates between 1995 and 2011-12 across the BMI categories. In particular for males, the increase was 11% for obese males, while only 5% for males of normal weight. However, the table above shows that the decrease in mean energy intake is relatively constant at around 11% for all BMI categories for males and varies between a decrease of 3% to an increase of 2% for females.

How much energy is potentially missing?

It is difficult from the available data to accurately estimate how much energy might be missing from the intakes reported by respondents in the 1995 NNS and the 2011-12 NNPAS. One approach in nutrition surveys to estimate the effect of under-reporting has been to look at the difference between overall mean energy intakes, and energy intakes after removal of LERs (i.e. the difference between all intakes and intakes for plausible reporters only). This approach assumes that removal of LERs is sufficient to remove under-reporting bias from the sample. This method shows increases in mean energy intakes of 12% for males and 15% for females in 2011-12 when the LERs are removed (see first table below).

However, given that the mean EI:BMR ratio for plausible energy reporters for males (1.50) and for females (1.49) in 2011-12 NNPAS are both below the conservative minimum energy requirement of 1.55 for a normally active but sedentary population11, this approach seems likely to underestimate the potential bias. As another method of estimating the bias, the mean amount of energy required for each individual to achieve an EI:BMR ratio of 1.55 is presented in the second table below.


Comparison of Mean Energy Intakes (kJ) for All Respondents (a), and after removal of Low Energy Reporters (b)


Males
Females

All Respondents
Plausible Respondents
Difference
% Change
All Respondents
Plausible Respondents
Difference
% Change

1995 NNS
11,384
12,100
716
6
7,795
8,612
817
10
2011-12 NNPAS
9,934
11,080
1,147
12
7,497
8,613
1,117
15

(a) Energy intakes presented are for all respondents aged 10 years and over. All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.
(b) Those respondents for whom an EI:BMR ratio was not calculated were also removed, so figures given are specifically for plausible energy reporters only. Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.

Estimation of Under-reporting Bias for Energy: Mean Energy to Achieve an EI:BMR of 1.55 (a) (b)


Males
Females

Mean Energy Intake (kJ)
Mean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55
Deficit as a % of reported energy
Mean Energy Intake (kJ)
Mean Energy Deficit (kJ) to Achieve an EI:BMR of 1.55
Deficit as a % of reported energy

1995 NNS
11,384
85
1
7,795
1,221
16
2011-12 NNPAS
9,934
1,672
17
7,497
1,573
21

(a) Energy intakes presented are for all respondents aged 10 years and over. All energy intakes include the contribution from fibre so will not match previously published figures for the 1995 NNS.
(b) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.
The results of this second table show that the mean increase in energy intake required to achieve an EI:BMR ratio of 1.55 for each individual is 17% for males and 21% for females.

The graph below shows that respondents in all BMI categories on average reported less than conservative minimum energy requirement with the largest deficit for obese females.

Graph Image: Mean reported energy intake over basal metabolic rate by BMI, 2011-12(a)(b)
(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.
(b) Line at EI:BMR = 1.55 indicates minimum average energy requirement for a normally active but sedentary population (not sick, disabled or frail elderly)

The graph below shows that for all age groups, males and females on average reported less than conservative minimum energy requirement.

Graph Image: Mean reported energy intake over basal metabolic rate by age, 2011-12 (a) (b)
(a) Respondents for whom a measured weight or height were not available, children under 10 years of age, and pregnant women were excluded from all analyses based on EI:BMR.
(b) Line at EI:BMR = 1.55 indicates minimum average energy requirement for a normally active but sedentary population (not sick, disabled or frail elderly)

Summary

In order to use this information to assist in the interpretation of data from the 2011-12 NNPAS and particularly in comparisons with the 1995 NNS, there are a few key points that should be noted.
  • It is likely that under-reporting is present in both surveys.
  • There appears to be an increase in the level of under-reporting for males between 1995 and 2011-12.
  • The level of under-reporting by female respondents also appears to have increased, but to a lesser extent than for males.
  • In order to achieve an EI:BMR ratio of 1.55 which is the amount required for a normally active but sedentary population, an increase in mean energy intake of 17% for males and 21% for females is required and greater increases are required for overweight and obese people than those of normal weight.
  • Given the association of under-reporting with overweight/obesity and consciousness of socially acceptable/desirable dietary patterns, under-reporting is unlikely to affect all foods and nutrients equally.

There is still further work that can be conducted in this area. In particular, the investigation into the impact of under-reporting on the change in consumption patterns of different foods can be expanded.


ENDNOTES

1 Macdiarmid J and Blundell J 1998, ‘Assessing dietary intake: Who, what and why of under-reporting’, Nutrition Research Reviews, 11, pp 231-253. doi:10.1079/NRR19980017, Available from <http://www.ncbi.nlm.nih.gov/pubmed/19094249>.
2 Moshfegh AJ, Rhodes DG, Baer DJ, Murayi T, Clemens JC, Rumpler WV, ... & Cleveland LE 2008, 'The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes', The American journal of clinical nutrition, 88(2), 324-332, Last accessed 05/05/2014, <http://ajcn.nutrition.org/content/88/2/324.full.pdf+html>.
3 ABS 2013, Australian Health Survey First Results, 2011-12 accessed online <https://www.abs.gov.au/ausstats/abs@.nsf/Lookup/E11CED5FB86D178ACA257AA30014C059?opendocument>.
4 Gibson RS 2005, Chapter 5: 'Measurement errors in dietary assessment', Principles of Nutritional Assessment Second Edition, Oxford University Press, pp.105-128.
5 Goldberg GR, Black AE, & Jebb SA et al 1991, ‘Critical evaluation of energy intake data using fundamental principles of energy physiology: 1 Derivation of cut-off limits to identify under-reporting’, European Journal of Clinical Nutrition, vol 45, pp.569-581.
6 ABS 1998, National Nutrition Survey: Nutrient Intakes and Physical Measurements, Australia 1995 (see Appendix 4).
7 Gibson RS 2005, Chapter 5: 'Measurement errors in dietary assessment', Principles of Nutritional Assessment Second Edition, Oxford University Press, p.168.
8 Gemming L et al. 2014, ‘Under-reporting remains a key limitation of self-reported dietary intake: an analysis of the 2008/09 New Zealand Adult Nutrition Survey’, European Journal of Clinical Nutrition, 68: 259-264, last accessed online 28 April 2014.
9 Maurer J, Taren LT, Teixeira PJ, Thomson CA, Lohman TG, Going SB, Houtkooper LB 2006, ‘The psychosocial and behaviour characteristics related to energy misreporting’, Nutrition Reviews, vol 64, no. 2, pp 53-66.
10 Black AE 2000, ‘The sensitivity and specificity of the Goldberg cut-off for EI:BMR for identifying diet reports of poor validity’, European Journal of Clinical Nutrition, vol. 54, pp395-404.
11 Black AE 2000, ‘Critical evaluation of energy intake using the Goldberg cut-off for energy intake:basal metabolic rate. A practical guide to its calculation, use and limitations’, International Journal of Obesity, vol. 24, pp1119-1130.
12 Gibson RS 2005, Principles of Nutritional Assessment Second Edition, Oxford University Press, p.121.
13 NHMRC 2006, Nutrient Reference Values for Australia and New Zealand, <http://www.nrv.gov.au/dietary-energy>, Last accessed 30/04/2014.






Previous PageNext Page