Critique 254: Adjusting for underreporting of alcohol intake in epidemiologic studies – 13 April 2022
Esser MB, Sherk A, Subbaraman MS, Martinez P, Karriker-Jaffe KJ, Sacks JJ, Naimi TS. Improving Estimates of Alcohol-Attributable Deaths in the United States: Impact of Adjusting for the Underreporting of Alcohol Consumption. J Stud Alcohol Drugs 2022;83:134-144. doi: 10.15288/jsad.2022.83.134.
Authors’ Abstract
Objective: Self-reported alcohol consumption in U.S. public health surveys covers only 30%-60% of per capita alcohol sales, based on tax and shipment data. To estimate alcohol-attributable harms using alcohol-attributable fractions, accurate measures of total population consumption and the distribution of this drinking are needed. This study compared methodological approaches of adjusting self-reported survey data on alcohol consumption to better reflect sales and assessed the impact of these adjustments on the distribution of average daily consumption (ADC) levels and the number of alcohol-attributable deaths.
Method: Prevalence estimates of ADC levels (i.e., low, medium, and high) among U.S. adults who responded to the 2011-2015 Behavioral Risk Factor Surveillance System (BRFSS; N = 2,198,089) were estimated using six methods. BRFSS ADC estimates were adjusted using the National Alcohol Survey, per capita alcohol sales data (from the Alcohol Epidemiologic Data System), or both. Prevalence estimates for the six methods were used to estimate average annual alcohol-attributable deaths, using a population-attributable fraction approach.
Results: Self-reported ADC in the BRFSS accounted for 31.3% coverage of per capita alcohol sales without adjustments, 36.1% using indexed-BRFSS data, and 44.3% with National Alcohol Survey adjustments. Per capita sales adjustments decreased low ADC prevalence estimates and increased medium and high ADC prevalence estimates. Estimated alcohol-attributable deaths ranged from approximately 91,200 per year (BRFSS unadjusted; Method 1) to 125,200 per year (100% of per capita sales adjustment; Method 6).
Conclusions: Adjusting ADC to reflect total U.S. alcohol consumption (e.g., adjusting to 73% of per capita sales) has implications for assessing the impact of excessive drinking on health outcomes, including alcohol-attributable death estimates.
Forum Comments
Assessment of alcohol intake: There is no ideal way of estimating the intake of alcohol of individuals, as no biologic measurement currently available accurately relates to a person’s ingestion. One-on-one interviews are one way that alcohol data can be collected, but is generally considered impractical for studies of large numbers of subjects. Usually self-administered questionnaires are used to assess the intake of alcoholic beverages in surveys and most large cohort studies.
There are obvious problems with individuals reporting their own intake of alcohol, and there is evidence that, overall, there is considerable underreporting of the amount consumed. As described by Forum member Skovenborg, reasons for such underreporting include the following:
- “Recall bias: difficulties in recall of drinking practices; the reported alcohol intake declines when the recall period increases.
- Variation in the alcohol content of a drink of beer, wine or spirits typically poured by the survey population sample associated with the challenging issue of drinker’s inability to accurately gauge their consumption in standard drinks.
- Social desirability bias: deliberate underreporting due to culturally determined socially desirable answers.
- Non-response bias: lack of response from people in the survey sample with a serious drinking problem and, for example, people with no telephone.
- Bias due to omission of irregular heavy drinking and special occasions drinking in survey questionnaires. In countries with Mediterranean-style drinking (frequent, regular drinking with meals), a quantity– frequency index questionnaire may correctly measure most of the alcohol consumption. In “dry” cultures with less frequent drinking with meals and occasional heavy episodic drinking, questions regarding special occasion drinking are indispensable.
- Beverage preference underreporting: In The Canadian Alcohol and Drug Use Monitoring Survey (CADUMS) 2008 to 2010, spirits consumption was underestimated by 65.94% compared with sales data, wine by 38.35% and beer by 49.02%. In the study of 39 European surveys by Kilian et al the lowest coverage was found for spirits consumption: 26.3% (CI 21.4-31.3).
- Non-differential underreporting of an exposure (e.g. alcohol consumption) may mask a true threshold effect as a dose-response relation and, if a true threshold effect exists, the threshold will be set at too low a level if the exposure is underreported (Kesmodel et al).
- Differential underreporting (e.g. disproportionately underreporting of alcohol consumption by heavy drinkers) would decrease the prevalence of high Average Daily Consumption (ADC) and increase the prevalence of medium/low ADC and spuriously increase the association of medium/low ADC and risk of alcohol-attributable disease and death. Estimates of drinking above recommended levels are likely to be disproportionately under-estimated (Boniface et al).
Methods proposed for “adjusting” self-reports of alcohol in epidemiologic studies: Forum member Skovenborg commented on the methods for adjustment of self-reported alcohol intake. “A variety of approaches have been used in an attempt to adjust the self-reported data to more realistic values; the one used in the present paper to “adjust” for actual intake involves comparisons of self-reports with “disappearance” of alcohol based on reported sales or tax records. This method assumes that all of the alcohol purchased gets consumed by the buyer or his friends, and none gets wasted. Data on conditions attributable to alcohol rely on prevalence estimates of alcohol use at various levels of consumption, and relative risks on the relationship between average daily consumption (ADC) and the risk of death from alcohol-related health conditions. However, self-reported alcohol consumption in U.S. public health surveys generally only accounts for 30%–60% of presumed consumption from per capita alcohol sales, based on tax and shipment data, resulting in conservative estimates of the alcohol-related public health impact.”
Skovenborg agrees that there may be substantial underreporting of alcohol consumption in surveys. “However, the issue of underreporting is very complicated and the methods to adjust for the underreporting are fraught with estimations, assumptions, generalizations and other methodological problems that leave the results of adjusting for underreporting open to question. The proportion of the ‘real’ consumption that is covered by surveys is known as coverage rate. While being the gold standard to determine the level of consumption in a country, per capita consumption cannot replace survey data because it does not tell us anything about the prevalence of alcohol use, patterns of consumption among drinkers of different demographic groups and variations of consumption levels in the population. This information is necessary for calculating the effects of alcohol consumption, however, and the self-report issue, which remain unresolved, is affected by many factors.
“The various reasons for underreporting in drinking surveys (that typically account for about half or even less of the national consumption) may have different effects on the degree of underreporting in surveys from different countries with different drinking cultures and different types of questions used to measure alcohol consumption. In a study of alcohol-consumption estimates in surveys in Europe, the coverage of sales estimates by surveys varied between 39% in Germany and 56% in France (Knibbe et al). Also, a recent review of 39 European surveys by Killian et al found large variation in coverage across 23 European countries with an average total coverage of 36.5% (CI 33.2-39.8). Variation among populations could markedly affect results of studies in which the same percentage is used for adjusting intake for all countries.
“Self-reported ADC in the 2011-2015 Behavioral Risk Factor Surveillance System (BRFSS), the basis for the present study, accounted for 31.3% coverage of per capita alcohol sales without adjustments. Among the six methods to adjust for underreporting, the authors adjusted to 73% of per capita sales to reflect total U.S. alcohol consumption, stating ‘The 73% multiplier was selected to match the coverage of per capita sales (from the AEDS) that is achieved in epidemiological cohort studies used to derive condition-specific relative risk estimates (Stockwell et al)’”
Attempts to adjust alcohol intake by other characteristics of individual subjects:. According to Stockwell et al. the issue of underreporting has not been well addressed in the published literature concerning epidemiological studies. He states: ‘Estimates of alcohol-related harm in a population calculated in estimation exercises such as the GBD study use the population-attributable fraction (PAF) method, i.e. the fractions of cases for a particular disease or injury attributable to an exposure of interest such as alcohol. PAF calculations require: (i) relative risk estimates for different diseases at various levels of consumption (which are based mainly on cohort studies) and (ii) prevalence estimates of various levels of exposure to consumption in the population to which to apply the relative risks (based on population surveys). There is a threat to validity, however, if the levels of under-reporting in these two types of studies (i.e. cohort and population surveys) are not comparable. To minimize this problem, current GBD methods raise survey-based estimates of total population exposure to alcohol to 80% of best estimates of age 15+ per capita alcohol consumption (Kehoe et al). This method assumes that the cohort studies used to estimate disease risk only underestimate alcohol consumption by 20%.’ Six methods of adjusting for underreporting in the BRFSS survey are tested in the present paper; adjusting to 73% of per capita alcohol sales to align with consumption reported among U.S. cohort study participants ADC prevalence estimates using per capita alcohol sales data was judged to be an appropriate and practical adjusting method.
Skovenborg continued: “The problem seems to be estimation of underreporting in cohort studies providing relative risk estimates for different disease at various levels of consumption. An analysis of 40 cohort studies from 18 countries published between 1980-2016 (Stockwell et al) found an overall coverage of WHO per capita consumption data of 61.71% calculated with baseline estimates of alcohol consumption. A methodological concern is the heterogeneity of the alcohol use methods employed by the cohort studies. And the fact that the risk estimates used in the present paper to calculate alcohol-attributable fractions of 58 alcohol-related condition are based on cohort data used in the Alcohol-Related Disease Impact application. The cohort study references in ARDI Methods present little or no information about underreporting. For example, Bagnardi et al stated: ‘An under-reporting of alcohol consumption in drinkers may partly explain the association with light alcohol drinking. In fact, alcohol consumption might be systematically underreported by both cases and controls (non-differential underreporting). This would lead to an overestimation of the RR for low doses’ In addition, it is hardly reassuring that a rating of overall confidence in the results of the Bagnardi et al’s review by Amstar (a critical appraisal tool for systematic reviews) was judged as critically low (Shea et al).
Methods used for “adjusting” for underreporting in reports from the Kaiser Permanente studies: Forum member Ellison pointed out that the authors of the present study do not mention what he believes is perhaps the most effective way of judging individual alcohol intake: making adjustments according to available data on other characteristics of subjects related to alcohol misuse. “Arthur Klatsky and his associates have described how numerous recorded characteristics of subjects suggesting alcohol misuse (admissions for alcohol intoxication, alcoholic cirrhosis, missing work due to alcohol misuse, etc.) may be used to separate “likely underreporters” from subjects who are “unlikely to be underreporters.” Their studies have shown clear relations between subjects in these classifications to important biological outcomes: risk of hypertension (Klatsky et al, 2006), total mortality (Klatsky et al, 2007), and risk of cancer (Klatsky et al, 2014). In fact, among subjects reporting ‘1 to 2 drinks/day,’ those deemed to be unlikely underreporters tend to show no increase in their risk of disease, while those classified as likely underreporters show increased risk, even at the same self-reported level of alcohol consumption. This suggests marked differences in underreporting of intake according to the usual level of drinking. Many epidemiologic studies of alcohol effects also have data on such characteristics of their subjects, presenting a potentially important method for estimating underreporting.”
Reviewer de Gaetano added: “I strongly support the method described by Klatsky and his associates for adjusting for underreporting of alcohol. If the analytic approaches presented by the authors of this paper indicate that people in the USA drink more than what they declare, this suggests that many may be consuming at higher levels than recorded, and ‘alcohol-attributable deaths’ in the USA could be higher than suggested by analyses based only on self-reported data. However, underreporting also suggests that the beneficial effects of moderate amounts of alcohol on cardiovascular disease risk may be associated with even higher amounts of alcohol than the amount reported in cohort studies. And, as Klatsky has clearly shown, the reported association of even ‘light to moderate’ intake with certain outcomes, such as cancer and total mortality, may actually be substantially less than that suggested by many studies.”
Forum member Lanzmann wrote: “The Klatzky studies at Kaiser Permanente were clear on the subject and have always seemed very important to me, but unfortunately have not received enough attention from epidemiologists and are not even cited in this paper. We all agree that, for many obvious reasons, there is often under-reporting of alcohol consumption. Thus, when we observe an inverse relationship between wine consumption and coronary mortality, for example with an RR of 0.8 for 1 glass/day and 0.7 for 2 glasses/day, the subjects may actually have consumed averages of 1.5 to 3 glasses/day, respectively. This means not only that wine consumption is correlated with coronary protection, but also that it is safe to drink more wine than we thought, perhaps even if wine drinkers seem to under-report less than others. And if we observe a correlation between the consumption of 2 glasses of alcohol and the risk of colon cancer, but the consumer actually drank 3 or 4 drinks per day, it would suggest that the risk observed for increased risk for reports of 2 glasses per day is overestimated. I also agree with remarks of others on attributable risk, which is used by the public authorities in European countries and in particular in France, to say arbitrary things about the number of alcohol-related deaths.”
Ellison added: “Overall, we agree that the application of the same formula for true underreporters and unlikely underreporters, as used in this paper, can lead to unreliable results. If an individual subject is a true underreporter, and their alcohol intake is greater than reported, their presence in a drinking category would tend to increase the risk of disease of subjects in that category inappropriately. (Such subjects should clearly have been placed in a higher category of alcohol consumption.) For subjects who are not underreporting their alcohol intake, it would suggest that the risk of disease for subjects at a given level of drinking may be inappropriately increased, whereas their actual alcohol intake would not have been likely to increase their risk of disease.”
Problems estimating alcohol-attributable fractions for many diseases: Skovenborg wrote: “The papers on alcohol-attributable fractions of various diseases, e.g. cancer, are very long and very complicated. Further, there are problems using sometimes outdated or biased data on disease entities when judging alcohol’s effects on health. Even though the present authors show gross underreporting of alcohol in most studies, it appears that they have used unadjusted data to estimate alcohol-attributable disease. The authors base their calculations of population attributable risk on the hypothetic assumption of a rather low degree of underreporting in the population studies that are used as foundation of relative risk associations between alcohol intake and disease.” Reviewer Waterhouse agreed: “I am concerned that the authors of this paper apparently applied a correction factor to the data on the calculated outcome of alcohol consumption, but did not apply any correction to the calculations used to estimate the impact of underreporting on the data for the health problems caused by alcohol consumption. In addition, this study assumes that there are only health problems associated with alcohol consumption and ignores the well documented health benefits associated with moderate alcohol consumption.”
Reviewer Djoussé added: “The concept of attributable fraction of an exposure to a specific disease is conditioned on a causal relation between exposure and outcome. Unfortunately, observational data may provide us with good/reasonable estimates of effects but fall short from providing evidence for causality. Mathematical modelling and exploration of various distribution forms of alcohol consumption are useful tools but cannot completely overcome shortcomings that are inherent to observation designs.”
To what extent does the present paper provide data for developing population guidelines regarding alcohol intake? Reviewer Skovenborg concludes his remarks: “To advise the public on ‘sensible’ limits of alcohol intake, methods are needed that properly rank individuals according to alcohol intake, and that also assess correctly the absolute level of intake and the pattern of drinking (e.g., regularly or only on weekends, with or without food, differently according to the type of beverage, differently according to the culture of their country, etc). With WHO including data from 40 cohort studies from different 18 countries, that requirement is hardly satisfied: such analyses cannot adjust for large differences in drinking habits that have clearly been demonstrated for different countries and cultures.” Reviewer Ellison added: “I believe that large cohort studies within specific populations, when properly adjusted for pattern of drinking and using an efficient method for identifying underreporting, may give a much better estimate of alcohol’s effects on health, and are a better choice for forming the basis for setting drinking guidelines for populations.”
Forum members agree that it is frustrating that data on the pattern of drinking are almost non-existing in the cohort data used in the Alcohol-Related Disease Impact application. It is unfortunate that this methodologic problem regarding estimates of alcohol-attributable deaths is ignored by the authors of the present paper. Skovenborg noted: “The relevance of drinking pattern is supported by a study by Gmel et al who modeled the effects of changes in aggregate consumption on mortality across countries using indicators of drinking patterns (including drinking with meals, frequency of drinking, drinking to intoxication, and percentage of abstainers in a population). The study showed that the more detrimental the general pattern of drinking in a country, the higher the impact of a change in alcohol consumption on all-cause mortality. As shown repeatedly in the Kaiser Permanente studies by Klatsky et al, data on other characteristics can be used to identify the likely level of underreporting of alcohol for individual subjects. Many other epidemiologic studies have similar data available that could provide individual adjustments for such factors; this could provide more reliable information for judging the overall effects of alcohol consumption on health.”
References from Forum Critique:
ARDI Methods. Centers for Disease and Prevention 2020 – retrieved from https://www.cdc.gov/alcohol/ardi/methods.html
Bagnardi V, Rota M, Botteri E, Tramacere I, et al. Alcohol consumption and site-specific cancer risk: a comprehensive dose–response meta-analysis. British Journal of Cancer 2015;112:580–593.
Boniface S, Kneale J, Shelton N. Drinking pattern is more strongly associated with under-reporting of alcohol consumption than socio-demographic factors: evidence from a mixed-methods study. BMC Public Health 2014;14:1297.
Doll R. The beneficial effects of alcohol on vascular disease. Addiction 1996;91:32-33.
Farchi G , Fidanza F, Giampaoli S, Mariotti S, Menotti A. Alcohol and survival in the Italian rural cohorts of the Seven Countries Study. Int J Epidemiol 2000;29:667-671.
Gmel G, Rehm J, Frick U. Methodological approaches to conducting pooled cross-sectional time series analysis: The example of the association between all-cause mortality and per capita alcohol consumption for men in 15 European states. European Addiction Research 2001;7:128–137.
Kehoe T, Gmel G, Shield KD, Gmel G, Rehm J. Determining the best population-level alcohol consumption model and its impact on estimates of alcohol-attributable harms. Population Health Metrics 2012;10:6.
Kesmodel U. Information bias in epidemiological studies with a special focus on obstetrics and gynecology. Acta Obstet Gynecol Scand 2018;97:417–423.
Kilian C, Manthey J, Probst C, Brunborg GS, et al. Why is per capita consumption underestimated in alcohol surveys? Results from 39 surveys in 23 European countries. Alcohol and Alcoholism 2020;55:554–563.
Klatsky AL, Gunderson EP, Kipp H, Udaltsova N, Friedman GD. Higher prevalence of systemic hypertension among moderate alcohol drinkers: an exploration of the role of underreporting. J Stud Alcohol 2006;67:421-428.
Klatsky AL, Udaltsova N. Alcohol drinking and total mortality risk. Ann Epidemiol 2007;17:S63-S67.
Klatsky AL, Udaltsova N, Li Y, Baer D, Tran N, Friedman GD. Moderate alcohol intake and cancer: the role of underreporting. Cancer Causes Control 2014;25:693-699.
Knibbe RA, Bloomfield K. Alcohol consumption estimates in Surveys in Europe: Comparability and sensitivity for gender differences. Substance Abuse 2001;22:23-38.
National Institute on Alcohol Abuse and Alcoholism. (2017). Alcohol Epidemiologic Data Directory. Retrieved from https://pubs.niaaa.nih.gov/publications/2017DataDirectory/2017DataDirectory.htm#1
Shea BJ, Reeves BC, Wells G, Thuku M, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.
Stockwell T, Zhao J, Macdonald S. WHO under-reports their alcohol consumption in telephone surveys and by how much? An application of the Yesterday Method in a national Canadian substance use survey. Addiction 2014;109:1657-1666.
Stockwell T, Zhao J, Sherk A, Rehm J, et al. Underestimation of alcohol consumption in cohort studies and implications for alcohol’s contribution to the global burden of disease. Addiction 2018;113:2245–2249.
Forum Summary
It has been demonstrated repeatedly that the total population alcohol consumption based on self-reports by subjects in surveys and epidemiologic studies is lower than the amount of alcohol sold or taxed (disappearance data) within the population. This is assumed to relate to marked underreporting of individuals of their consumption, sometimes by one half or more. A key problem in attempting to adjust the self-reported values to more closely match the total alcohol disappearance data is that everyone does not underreport their intake, or may do so by different degrees.
The present study was based on data from US adults who responded to the 2011-2015 Behavioral Risk Factor Surveillance System (BRFSS; N = 2,198,089). The authors tested six methods for adjusting the self-reported data to provide what they hoped to be a more accurate value; some methods included adjustments based on differences in consumption by age and sex subgroups. From an unadjusted value of self-reported consumption of only 31.3% of that indicated by disappearance data, the various methods provided adjusted values ranging from 36.1% to 73%. The authors then used higher values to estimate the effect on alcohol-attributable deaths, using a population-attributable fraction approach.
While applauding the authors for attempting to obtain better estimates of alcohol consumption, Forum members were surprised that the authors did not even mention the method for identifying underreporting of individual subjects described by Klatsky and colleagues in the Kaiser Permanente Studies. The results of their method have been demonstrated to relate closely to alcohol’s effects on the risk of cancer, hypertension, and total mortality. Their method includes, in addition to self-reported data on alcohol consumption, adjustments for a variety of health conditions known to relate to excessive alcohol exposure, including alcoholic liver disease, hospitalizations for intoxication, mental problems associated with heavy drinking, etc. The presence of these conditions not only suggested that such subjects markedly underreported their intake (the exposure), but they also had much higher risk of adverse health effects (the outcome). When these conditions were not present at any of the study visits of subjects (allowing such subjects to be classified as being unlikely to be underreporting their alcohol consumption), adverse effects of low to moderate alcohol intake were generally not seen or their occurrences were markedly less frequent. This adjustment approach seems to be the only one previously described that gives reasonable results for individuals, and is clearly associated with the future health outcomes of subjects (and we realize that the occurrence of certain diseases or death happens for individual subjects, not for the population).
The Forum also raised questions about using “alcohol-attributable risk”. The authors base their calculations of population attributable risk on the hypothetic assumption of a rather low degree of underreporting in the population studies that are used as foundation of relative risk associations between alcohol intake and disease. Thus, risk is based mainly on self-reports of intake, even though their present paper indicates that adjustment for underreporting is so important. Further, throughout the paper, it appears that the authors tend to focus only on the adverse effects of alcohol without taking into regard the beneficial effects of light-to-moderate drinking, especially the regular consumption of wine with meals, which is found almost universally to be associated with lower risk of cardiovascular disease, diabetes, and total mortality.
While appropriate adjustment for underreporting could greatly improve the results of epidemiologic studies on alcohol and health, the goal would be to develop methods of determining the risk of individual subjects by including data on their own pattern of drinking and not assuming that the same adjustment formula applies to everyone in a population. By being able to identify subjects who underreport their intake, it would lead to some people changing categories of intake, and should result in estimates of either a lower risk, or greater risk, of adverse health effects of alcohol for that category of drinking. Further, more accurate and updated assessments of future health conditions that may be adversely (or beneficially) related to alcohol consumption should improve results and provide better data for setting drinking guidelines.
* * * *
Contributions to this critique by the International Scientific Forum on Alcohol Research were provided by the following members:
Erik Skovenborg, MD, specialized in family medicine, member of the Scandinavian Medical Alcohol Board, Aarhus, Denmark
R. Curtis Ellison, MD, Professor of Medicine, Emeritus; Section of Preventive Medicine & Epidemiology, Boston University School of Medicine, Boston, MA, USA
Andrew L. Waterhouse, PhD, Department of Viticulture and Enology, University of California, Davis, USA
Harvey Finkel, MD, Hematology/Oncology, Retired (Formerly, Clinical Professor of Medicine, Boston University Medical Center, Boston, MA, USA)
Creina Stockley, PhD, MSc Clinical Pharmacology, MBA; Principal, Stockley Health and Regulatory Solutions; Adjunct Senior Lecturer, The University of Adelaide, Adelaide, Australia
Giovanni de Gaetano, MD, PhD, Department of Epidemiology and Prevention, IRCCS Istituto Neurologico Mediterraneo NEUROMED, Pozzilli, Italy
Ramon Estruch, MD, PhD, Hospital Clinic, IDIBAPS, Associate Professor of Medicine, University of Barcelona, Spain
Luc Djoussé, MD, DSc, Dept. of Medicine, Division of Aging, Brigham & Women’s Hospital and Harvard Medical School, Boston, MA, USA
Dominique Lanzmann-Petithory, MD, PhD, Nutrition Geriatrics, Hôpital Emile Roux, APHP Paris, Limeil-Brévannes, France
Pierre-Louis Teissedre, PhD, Faculty of Oenology–ISVV, University Victor Segalen Bordeaux 2, Bordeaux, France