Criterion validity of self-reports of alcohol, cannabis, and methamphetamine use among young men in Cape Town, South Africa

Accurate screening for substance use is important for research, public health, and clinical work. This study examines how well self-reports of usage of three substances—alcoholic beverages, cannabis (known locally as "dagga"), and methamphetamine (known locally as "tik")—predict physiological tests of substance use among young men living in Cape Town, South Africa. The study was conducted in advance of a community-level intervention for substance use and other risky behaviors. In this setting, as in any program to treat or prevent substance use, it is important to have valid measures of substance use in order to evaluate individual outcomes and the overall intervention. Self-report is an attractive and widely used option because it is inexpensive, non-invasive, quick, and convenient, but self-report is not guaranteed to be valid. In particular, stigmatized behavior such as substance use may be underreported due to social desirability bias: people can be understandably reluctant to admit stigmatized behavior. What's more, people may simply misremember their behavior, especially if their memory has been adversely affected by substance use, as by an alcohol-induced blackout (White, 2003). To this end, we estimated the criterion validity of self-report by comparing it to physiological tests.

Previous research of this kind suggests that self-report is often accurate, but accuracy can vary by population and by the substance being examined. We will focus on "honesty rates", the proportion of people who admit use among all people for whom a physiological test detected use. (In the language of binary classification, the honesty rate is called the "hit rate", "sensitivity", "recall", or "true positive rate"; it is equal to 1 minus the false negative rate. Despite our use of the word "honesty", honesty rates can of course be influenced by memory errors as well as deliberate misrepresentation.) Honesty for alcohol has received the most research attention, albeit only in clinical or treatment settings. High honesty rates (≥ 83%) have been found when using breath, urine, or hair tests in Sweden (Dahl, Hammarberg, Franck, & Helander, 2011), Australia (Bonevski, Campbell, & Sanson-Fisher, 2010), and Cape Town (Cherpitel et al., 2007; Peden, van der Spuy, Smith, & Bautz, 2000; Kader, Seedat, Koch, & Parry, 2012). An exception is a urine-based study of Swedish opioid addicts in methadone maintenance treatment, who were only 65% honest about alcohol (Helander et al., 1999).

For cannabis, urine tests have found a broader range of honesty rates, perhaps partly due to the use of arrestee populations. American adolescents in cannabis dependence treatment were 77% honest (Buchan, Dennis, Tims, & Diamond, 2002), American arrestees were 64% honest (Lu, Taylor, & Riley, 2001), South African arrestees were 54% honest (Plüddemann & Parry, 2003), and Cape Town trauma center patients were 40% honest (Peden et al., 2000). The lowest honesty rate, 22%, was observed in a roadside survey of Belgian drivers that used a saliva test (Van der Linden, Silverans, & Verstraete, 2014).

We know of two previous studies reporting honesty rates for methamphetamine. The previously mentioned sample of American arrestees (Lu et al., 2001) obtained an honesty rate of 56% for a urine test. A study of admissions to a California emergency room (Lee, Vivier, & Diercks, 2009) for chest pain (which is associated with methamphetamine use) found an honesty rate of 50% for a urine test.

Why might honesty differ by substance? One might speculate that honesty is affected by perceptions of the stigma and potential harm associated with a substance. Panels of experts (Nutt, King, Saulsbury, & Blakemore, 2007, Nutt, King, & Phillips, 2010) have rated alcohol as more harmful overall than amphetamines, and amphetamine more harmful than cannabis. A survey of young adults in Manhattan (Palamar, Kiang, & Halkitis, 2012) found that they stigmatized cannabis (mean rating 2.11 on a 5-point scale) slightly less than amphetamine (mean rating 2.71). A survey of American college students (Spigner, Hawkins, & Loren, 1993) found that 78% saw regular use of amphetamines as a "great risk", while 75% said the same of daily drinking, 60% said the same of cannabis use, and 43% said the same of weekend drinking. Overall, it seems likely that people will be less honest about their amphetamine use than their cannabis or alcohol use. The relative stigma of cannabis and alcohol is less clear. As for memory, chronic use of any of alcohol (Kopelman, 1995), cannabis (Solowij & Battisti, 2008), or methamphetamine (Nordahl, Salo, & Leamon, 2003) is associated with memory impairment. However, acute methamphetamine use can improve memory in at least some cases (Soetens, Casaer, D'Hooge, & Hueting, 1995), so methamphetamine reports may be less prone to memory errors than alcohol or cannabis reports.

In South Africa, as in many other jurisdictions, alcohol is legal to produce and to sell to any person who is at least 18 years old (South Africa, 2004), whereas cannabis and methamphetamine are illegal to possess or use (South Africa, 2003). People in Cape Town asked for their attitudes about a hypothetical addict described in a vignette (Sorsdahl, Stein, & Myers, 2012) gave mostly the same responses regardless of the substance the character was addicted to, including 64% agreeing that the addict's problems betrayed a weak character. In fact, they seemed to stigmatize addiction more than schizophrenia or panic disorder. However, they saw alcoholics as slightly more dangerous (mean rating 6.5 on a 9-point scale) than cannabis (6.0), methamphetamine (6.0), and even heroin (6.1) addicts. They also described a male alcoholic as needing to be coerced into treatment (6.6) more than a female alcoholic (5.8). Among workers in 11 South African mines, 97% agreed that alcohol use can lead to mining accidents, and 86% said the same for cannabis (Pick et al., 2003). A survey of methamphetamine users in a township near Cape Town (Meade et al., 2015) found that 90% were interested in getting treatment, but only 10% had ever received treatment. Cannabis seems to rarely be used as an alternative treatment for HIV: among HIV-positive people in KwaZulu-Natal, elsewhere in South Africa, just 4% said they used cannabis (Peltzer, Preez, Ramlagan, & Fomundam, 2008). Overall, South Africans appear to regard all three substances with caution, with no strong tendency to see them as differentially dangerous or stigmatizing, even though the law favors alcohol.

This study has two novel features that are useful for advancing the study of substance self-report validity. One, it considers a vulnerable population in need of intervention for substance use (young men in Cape Town) in a less formal setting than clinics or police stations, where honesty may well be different. Two, it examines alcohol, cannabis, and methamphetamine use all in one sample, allowing for direct comparisons of honesty rates. Kader et al. (2012) also considered all three substances in one sample, but due to their sample sizes (around 30), only alcohol had more than one positive test.

Men were recruited from 24 neighborhoods of Cape Town (the protocol is described in detail in Rotheram-Borus et al., 2017). The institutional review boards of the University of California, Los Angeles (IRB#14-001587) and Stellenbosch University (N14/08/116) approved the study protocol. In each neighborhood, a recruiter flipped a coin onto a map of the neighborhood to decide what dwelling to start with, then went from one dwelling to the next in concentric circles. People were invited to participate in the study if they were male, were aged 18 to 29 years, had slept in the dwelling in question on at least four nights a week over the past two months, spoke Xhosa or English, did not appear to be intoxicated, and were able to understand the recruiter. The recruiter continued until 50 men in the neighborhood had agreed to participate. The refusal rate was below 5%. Men completed consent forms before any assessment.

Homes in these areas are typically too small to allow for confidential interviewing. Hence, men were transported to a local storefront research study space for interviewing. Trained study personnel conducted a one-hour interview with each man, recording responses on mobile phones. The interviews began in October 2015 and are currently ongoing. The interviews covered a variety of topics including basic demographic information, substance use, and contact with the criminal justice system. In this paper, we analyze the responses to three interview questions: "How many drinks containing alcohol have you had in the last three days?" (with responses dichotomized as zero vs. nonzero), "Have you used dagga [i.e., cannabis] in the last two days?", and "Have you used tik [i.e., methamphetamine] in the last two days?". We use the data up to but not including February 2017.

Men also completed urine-based rapid diagnostic tests (RDT) sold by CLIAwaived Inc. (San Diego, CA), one for each substance. The alcohol RDT (ETG-1000) detects ethyl glucuronide as low as 500 ng/mL. The cannabis RDT (CLIA-SDDT-10) detects 11-nor-Δ⁹-tetrahydrocannabinol-9-carboxylic acid as low as 50 ng/mL. The methamphetamine RDT (CLIA-SDDT-16) detects methamphetamine as low as 500 ng/mL. Hence, these RDTs could detect alcohol use in the last 80 hours (over 3 days), cannabis use in the last 3 days (possibly up to 7 days, or 30 days in chronic users), and methamphetamine use in the last 3 days.

Some men have received the baseline assessment (including the interview and the RDTs) more than once, because delays in beginning the intervention led to the need to update the baseline information. In this paper, we use only the first assessment of each subject.

In a pilot study of this population (Rotheram-Borus et al., 2016), only 17% of men reported ever using methaqualone (Mandrax) and 16% reported ever using cocaine, compared to 23% for methamphetamine; given these low rates, methaqualone and cocaine were not considered in this study.

Table I reviews the sample. We obtained data for 1,155 men. The men were generally young (median age 22 years) and poor (52% earned R 500 or less per month). Substantial minorities had been arrested (34%) or joined a gang (22%) at least once. Most of the men said they drank (76%) or had used cannabis at least once (75%), whereas a minority said they had used methamphetamine at least once (27%).

Table I. Properties of the sample. Except for sample size, these variables are all self-reported.

Sample size	1,155
Age, years, median	22
Years of schooling, <9	13%
Years of schooling, 9–12	87%
Years of schooling, >12	1%
Monthly income ≤ R 500 (32 USD)	52%
Ever held a job	70%
Ever been arrested	34%
Ever been jailed	6%
Ever joined a gang	22%
Drinks alcohol	76%
Ever used cannabis	75%
Ever used methamphetamine	27%

Table II shows the confusion matrices of self-report used to predict RDTs. Overall, RDTs detected the respective substances (alcohol, cannabis, and methamphetamine) in 32%, 52%, and 22% of men. (Multiple substance use was common, with 26% of men having two RDTs positive and 5% having all three positive.) By contrast, 29%, 37%, and 11% of men reported use. The accuracies (i.e., proportions agreement, or consistencies) were 77%, 83%, and 88%. These compare favorably to the base rates of the modal class (i.e., for each substance, either the proportion of men with positive RDTs or the proportion with negative RDTs, whichever is larger), which are 68%, 52%, and 78%. Recall, however, that the time horizons asked about by the self-report questions did not exactly match the temporal thresholds of the RDTs.

We see that among men who denied use, only 18%, 25%, and 13% actually did use according to the RDTs. Contrapositively, among men for whom RDTs detected use, only 61%, 70%, and 48% admitted use. These last three figures are what we described in the introduction as "honesty rates". Jeffreys 95% confidence intervals for the honesty rates are [56%, 66%], [67%, 74%], and [42%, 54%].

To compare honesty between the three substances, we can examine the log odds ratios of each pair of honesty rates, and compute 95% confidence intervals with the percentile bootstrap. We find that men were 0.42 ([0.15, 0.69]) more honest for cannabis than alcohol, 0.53 ([0.21, 0.86]) more honest for alcohol than methamphetamine, and 0.95 ([0.67, 1.23]) more honest for cannabis than methamphetamine.

Table II. Confusion matrices for each of the three substances, comparing self-report to rapid diagnostic tests. Percentages are out of the column totals.

	Rapid diagnostic test
	Alcohol		Cannabis		Methamphetamine
	Negative	Positive	Negative	Positive	Negative	Positive
Self-report: no	671 (85%)	144 (39%)	545 (97%)	177 (30%)	897 (100%)	133 (52%)
Self-report: yes	116 (15%)	224 (61%)	14 (3%)	419 (70%)	3 (0%)	122 (48%)

It is also useful to examine RDT results for men who stated that they did not drink or that they had never used cannabis or methamphetamine. Among alleged teetotalers, 6% tested positive for alcohol. Among those who stated they had never used cannabis or methamphetamine, 4% and 6% tested positive, respectively.

Treating RDTs as the gold standard of whether men actually used a substance, we found that self-report was generally accurate. For each of the three substances, self-report matched biomarkers in at least 75% of men, and the accuracy was substantively higher than the base rate (i.e., the proportion of the RDT's modal outcome). At the same time, self-report missed many positive cases: honesty rates were 61% for alcohol, 70% for cannabis, and 48% for methamphetamine. The men were moderately more honest about cannabis than alcohol, and in turn moderately more honest about alcohol than methamphetamine.

These results are roughly comparable to those of previous studies. Compared to the previous samples discussed in the introduction (Bonevski et al., 2010; Buchan et al., 2002; Cherpitel et al., 2007; Dahl et al., 2011; Helander et al., 1999; Kader et al., 2012; Lee et al., 2009; Lu et al., 2001; Peden et al., 2000; Plüddemann & Parry, 2003; Van der Linden et al., 2014), our sample was especially honest about cannabis, especially dishonest about alcohol, and typical about methamphetamine. It may be surprising that the men were more honest about cannabis than about alcohol, since in South Africa, as in most of the world, alcohol is legal whereas cannabis is not. But as discussed in the introduction (Nutt et al., 2007; Nutt et al., 2010; Spigner et al., 1993; Sorsdahl et al., 2012), attitudes, among experts and laypeople alike, may hold drinking to be as bad as cannabis or worse. Recent successes by movements to legalize cannabis, such as Colorado Amendment 64 (Healy, 2014) and California Proposition 64 (McGreevy, 2016) in the US, suggest that law is beginning to change in order to conform to these attitudes. As for methamphetamine, while the honesty rate we obtained was typical of that of previous studies, its low value compared to that of the other substances is odd in light of its possible memory-boosting effects (Soetens et al., 1995) and how the respondents of Sorsdahl et al. (2012) perceived it as no more dangerous than alcohol. This said, Sorsdahl et al. asked for opinions about addicts specifically; one might imagine that without the qualification of addiction, methamphetamine use is stigmatized more than drinking.

A limitation of our study, as with most studies that seek to validate substance self-report, is that physiological tests are not (and cannot be) perfectly accurate. Hence, some of the men with a positive RDT who denied use may have been telling the truth. Another limitation is that the time horizons of the self-report questions and the RDTs were not equal. In particular, our RDT for cannabis could detect use as long ago as 30 days, whereas we asked about use in the past 2 days. This might be expected to produce a negatively biased estimate of the honesty rate for cannabis, but the fact that cannabis had the highest honesty rate of the three substances suggests that the mismatch was not a serious problem in practice. Finally, because of the consent process for the study, the men knew while they were answering self-report questions that they would later undergo physiological drug tests, if they hadn't already. Thus, they may have been more honest than they would have been in a study without physiological drug tests.

Future research of substance use in this population should be cognizant of the honesty rates obtained here. Most drinkers and users of cannabis can be expected to admit their use, but far from all of them: 61% and 70%, respectively. The case for methamphetamine is worse: self-report will be able to detect only a minority of users (48%), and thus is unlikely to be sufficient, especially for small samples.

One possibly striking characteristic of our sample is its poverty. Most of the men had an income of R 500 or below, well under the R 779 upper-bound poverty line estimated for 2014 in South Africa (Statistics South Africa, 2015), which is the amount required to purchase both enough food for good health and essential non-food items, without having to sacrifice either. However, the same poverty criterion is met by 54% of South Africans (Statistics South Africa, 2015), so our sample's poverty is not unusual.

Bonevski, B., Campbell, E., & Sanson-Fisher, R. W. (2010). The validity and reliability of an interactive computer tobacco and alcohol use survey in general practice. Addictive Behaviors, 35(5), 492–498. doi:10.1016/j.addbeh.2009.12.030

Buchan, B. J., Dennis, M. L., Tims, F. M., & Diamond, G. S. (2002). Cannabis use: Consistency and validity of self-report, on-site urine testing and laboratory testing. Addiction, 97(Suppl1), 98–108. doi:10.1046/j.1360-0443.97.s01.1.x

Cherpitel, C. J., Ye, Y., Bond, J., Borges, G., Macdonald, S., Stockwell, T., … Giesbrecht, N. (2007). Validity of self-reported drinking before injury compared with a physiological measure: Cross-national analysis of emergency-department data from 16 countries. Journal of Studies on Alcohol and Drugs, 68(2), 296–302. doi:10.15288/jsad.2007.68.296

Dahl, H., Hammarberg, A., Franck, J., & Helander, A. (2011). Urinary ethyl glucuronide and ethyl sulfate testing for recent drinking in alcohol-dependent outpatients treated with acamprosate or placebo. Alcohol and Alcoholism, 46(5), 553–557. doi:10.1093/alcalc/agr055

Healy, J. (2014, January 1). Up early and in line for a marijuana milestone in Colorado. The New York Times. Retrieved from http://www.nytimes.com/2014/01/02/us/colorado-stores-throw-open-their-doors-to-pot-buyers.html

Helander, A., von Wachenfeldt, J., Hiltunen, A., Beck, O., Liljeberg, P., & Borg, S. (1999). Comparison of urinary 5-hydroxytryptophol, breath ethanol, and self-report for detection of recent alcohol use during outpatient treatment: A study on methadone patients. Drug and Alcohol Dependence, 56(1), 33–38. doi:10.1016/S0376-8716(99)00007-1

Kader, R., Seedat, S., Koch, J. R., & Parry, C. D. (2012). A preliminary investigation of the AUDIT and DUDIT in comparison to biomarkers for alcohol and drug use among HIV-infected clinic attendees in Cape Town, South Africa. African Journal of Psychiatry, 15(5), 346–351. doi:10.4314/ajpsy.v15i5.43

Kopelman, M. D. (1995). The Korsakoff syndrome. British Journal of Psychiatry, 166(2), 154–173. doi:10.1192/bjp.166.2.154

Lee, M. O., Vivier, P. M., & Diercks, D. B. (2009). Is the self-report of recent cocaine or methamphetamine use reliable in illicit stimulant drug users who present to the Emergency Department with chest pain? Journal of Emergency Medicine, 37(2), 237–241. doi:10.1016/j.jemermed.2008.05.024

Lu, N. T., Taylor, B. G., & Riley, K. J. (2001). The validity of adult arrestee self-reports of crack cocaine use. American Journal of Drug and Alcohol Abuse, 27(3), 399–419. doi:10.1081/ADA-100104509

McGreevy, P. (2016, November 8). Voters legalize pot in California. Here's what will happen next. Los Angeles Times. Retrieved from http://www.latimes.com/politics/la-pol-ca-proposition-64-california-legalizes-marijuana-snap-20161108-story.html

Meade, C. S., Towe, S. L., Watt, M. H., Lion, R. R., Myers, B., Skinner, D., … Pieterse, D. (2015). Addiction and treatment experiences among active methamphetamine users recruited from a township community in Cape Town, South Africa: A mixed-methods study. Drug and Alcohol Dependence, 152, 79–86. doi:10.1016/j.drugalcdep.2015.04.016

Nordahl, T. E., Salo, R., & Leamon, M. (2003). Neuropsychological effects of chronic methamphetamine use on neurotransmitters and cognition: A review. Journal of Neuropsychiatry and Clinical Neurosciences, 15(3), 317–325. doi:10.1176/jnp.15.3.317

Nutt, D. J., King, L. A., & Phillips, L. D. (2010). Drug harms in the UK: A multicriteria decision analysis. Lancet, 376(9752), 1558–1565. doi:10.1016/S0140-6736(10)61462-6

Nutt, D., King, L. A., Saulsbury, W., & Blakemore, C. (2007). Development of a rational scale to assess the harm of drugs of potential misuse. Lancet, 369(9566), 1047–1053. doi:10.1016/S0140-6736(07)60464-4

Palamar, J. J., Kiang, M. V., & Halkitis, P. N. (2012). Predictors of stigmatization towards use of various illicit drugs among emerging adults. Journal of Psychoactive Drugs, 44(3), 243–251. doi:10.1080/02791072.2012.703510

Peden, M., van der Spuy, J., Smith, P., & Bautz, P. (2000). Substance abuse and trauma in Cape Town. South African Medical Journal, 90(3), 251–255. Retrieved from https://www.researchgate.net/profile/Margaret_Peden/publication/12465958/links/0912f50ae1d8ca4db8000000.pdf

Peltzer, K., Preez, N. F., Ramlagan, S., & Fomundam, H. (2008). Use of traditional complementary and alternative medicine for HIV patients in KwaZulu-Natal, South Africa. BMC Public Health, 8(255). doi:10.1186/1471-2458-8-255

Pick, W., Naidoo, S., Ajani, F., Onwukwe, V., Hansia, R., & Bielu, O. (2003). Prevalence of alcohol and cannabis use and reported knowledge, attitudes and practice regarding its relationship with health and safety on mines in South Africa. Council for Scientific and Industrial Research. Retrieved from http://hdl.handle.net/10204/1277

Plüddemann, A., & Parry, C. D. H. (2003). A short report: Self-reported drug use vs. urinalysis in a sample of arrestees in South Africa. Drugs: Education, Prevention and Policy, 10(4), 379–383. doi:10.1080/0968763031000102626

Rotheram-Borus, M. J., Tomlinson, M., Durkin, A., Baird, K., DeCelles, J., & Swendeman, D. (2016). Feasibility of using soccer and job training to prevent drug abuse and HIV. AIDS and Behavior, 20(9), 1841–1850. doi:10.1007/s10461-015-1262-0

Rotheram-Borus, M. J., Tomlinson, M., Mayekiso, A., Bantjes, J., Swendeman, D., Harris, D. M., & Weiss, R. E. (2017). Gender-specific HIV and substance abuse prevention strategies for South African men: Study protocol for a randomized controlled trial. Trials. Submitted.

Soetens, E., Casaer, S., D'Hooge, R., & Hueting, J. E. (1995). Effect of amphetamine on long-term retention of verbal material. Psychopharmacology, 119(2), 155–162. doi:10.1007/BF02246156

Solowij, N., & Battisti, R. (2008). The chronic effects of cannabis on memory in humans: A review. Current Drug Abuse Reviews, 1(1), 81–98. doi:10.2174/1874473710801010081

Sorsdahl, K., Stein, D. J., & Myers, B. (2012). Negative attributions towards people with substance use disorders in South Africa: Variation across substances and by gender. BMC Psychiatry, 12(101), 1–8. doi:10.1186/1471-244X-12-101

South Africa. (2003). Department of health: Schedules. Government Gazette of South Africa, 454(24727), 3–80. Retrieved from https://web.archive.org/web/20170216173236/http://www.gov.za/sites/www.gov.za/files/24727a.pdf

South Africa. (2004). Liquor Act 59 of 2003. Government Gazette of South Africa, 466(26294), 1–102. Retrieved from https://www.thedti.gov.za/business_regulation/docs/nla/act2003.pdf

Spigner, C., Hawkins, W. E., & Loren, W. (1993). Gender differences in perception of risk associated with alcohol and drug use among college students. Women and Health, 20(1), 87–97. doi:10.1300/J013v20n01_06

Statistics South Africa. (2015). Methodological report on rebasing of national poverty lines and development of pilot provincial poverty lines. Author. Retrieved from http://web.archive.org/web/20160220222420/http://beta2.statssa.gov.za/publications/Report-03-10-11/Report-03-10-11.pdf

Van der Linden, T., Silverans, P., & Verstraete, A. G. (2014). Comparison between self-report of cannabis use and toxicological detection of THC/THCCOOH in blood and THC in oral fluid in drivers in a roadside survey. Drug Testing and Analysis, 6(1, 2), 137–142. doi:10.1002/dta.1517

White, A. M. (2003). What happened? Alcohol, memory blackouts, and the brain. Alcohol Research and Health, 27(2), 186–196. Retrieved from https://pubs.niaaa.nih.gov/publications/arh27-2/186-196.htm