The issue of evidence being cited by representatives from companies has become increasingly important. This is because of the growth of public consultations to collate evidence before new laws are passed.
We examined a selection of submissions that big companies made to South Africa’s National Treasury during a consultation process on the country’s proposed new tax on sugar. Chief among these were Coca-Cola South Africa, the largest soft drinks manufacturer in the country, the Beverage Association of South Africa, the business association for the soft drinks sector in South Africa, and the American Chamber of Commerce in South Africa, a collective voice of US transnational corporations in South Africa.
Our analysis showed that the evidence cited by the three organisations was either not evidence at all, or had been twisted to suit the industry’s narrative.
The analysis
Our analysis built on a method used to evaluate evidence use by tobacco companies. There are several components to the method. The most important centres on something called backward-mapping. This involves comparing evidence presented in submissions with supporting sources and examining how they have been used.
To the casual reader, most of the claims that the industry made would have looked credible. They were often supported by references to peer-reviewed journals, data produced by respected organisations, such as the United Nations’ Food and Agriculture Organisation, and sophisticated-looking economic impact studies which had been commissioned by the industry.
By using the backward-mapping technique we were able to show that the evidence provided didn’t stand up to scrutiny.
For example, we found that in some cases industry representatives simply made things up. The American Chamber of Commerce in South Africa cited a 2013 report by Oxford Economics to support the claim that the impact of sugar sweetened beverages “on health outcomes is uncertain and unproven”.
But we found that the report didn’t in fact examine the relationship between sugar sweetened beverages and health.
Coca-Cola and the Beverage Association of South Africa also used what’s referred to as the “tweezers method” to get their case across. This involved changing the intended meaning of original text in peer reviewed journals by picking phrases out of context.
For example, Coca-Cola reported that a review by two researchers from Harvard School of Public Health had
concluded that there is limited evidence that consumers do not reduce their calorie intake to offset calories consumed in liquid form.
These comments were aimed at refuting the idea that someone’s overall daily energy intake was likely to go down if they lowered their intake of sugary drinks. The inference was that the research had found some evidence that people eat more to compensate for reduced sugary drink intake.
In fact the Harvard review had used “limited evidence” to highlight the dearth of studies on the issue. Moreover, the review had indicated that existing findings indicated that intake of sugary drinks may not suppress intake of solid foods to the level needed to maintain energy balance.
Another method we identified was “data dredging”.
Coca-Cola and the Beverage Association of South Africa drew heavily on Food Balance Sheets produced by the Food and Agriculture Organisation to argue that sugar wasn’t responsible for the rise in obesity in South Africa. They argued that sugar consumption had declined over time in contrast to the consumption of vegetable oils and cereals which they claimed accounted for the rise in average daily energy intake among South Africans. According to them, this was the biggest cause of obesity.
But they’d specifically selected a time period (1991-2011) that would illustrate this. They used Food and Agriculture Organisation balance sheets for 1991 as the starting point – a year in which the highest per capita daily supply of sugar was recorded over 25 years. In fact sugar consumption has risen – and vegetable oils and cereals have fallen – in both the 10 and 20-year period prior to the most recent reported figures (2013).
The soft drinks industry had commissioned the British-based research consultancy, Oxford Economics to model the potential impact of the tax on employment, tax revenue, and gross domestic product. Oxford Economics reported that the tax could lead to between 60,600 and 70,700 job losses. In their submission the beverage association and Coca-Cola pushed these numbers up to between 62,000 and 72,000.
Other aspects of Oxford Economics’ estimates raised questions too. It claimed that a key study that underpinned their estimates had reported that, “drinkers of sugar-sweetened beverages (were) unlikely to switch to bottled water” and that
other studies (had) not found statistically robust evidence that people switch from sugar-sweetened beverages to water when the price of sugar-sweetened beverages increase.
In fact, the study had made no explicit reference to bottled water, but had in fact observed that
other studies have shown that the demand for tea and coffee, as well as water goes up with sugar sweetened beverages price increases.
Vigilance needed
The above examples represent a small selection of the techniques we identified. Taken in isolation they may not seem that serious. However, the key to understanding the problem is to consider them together, as an alternate system of “agnotological” reasoning, which deliberately aims to produce ignorance and doubt about scientific evidence.
Our work highlights the degraded nature of business evidence and underlines the need to rethink the elevated position sometimes given to big corporations’ views in public health policy conflicts.
