False confidence? Cocoa research challenges reliability of sustainability audits

A study of cocoa farms in Côte d’Ivoire, the world’s largest cocoa-producing country, has found that sustainability monitoring can be vulnerable to manipulated data. The researchers say their findings raise serious questions about how reliably companies and certification schemes can verify claims about the environmental and social standards behind products such as chocolate.

Sustainability audits are supposed to give food and beverage companies confidence that the claims they make about their supply chains can withstand scrutiny. However, the ETH Zurich research into cocoa farming suggests the audit process itself can create an incentive to produce the “right” answer.

The findings could have particular significance for companies preparing to comply with the EU Deforestation Regulation, which requires companies to demonstrate that commodities, including cocoa, sold in the EU are not linked to deforestation or forest degradation worldwide.

Cocoa sustainability under scrutiny

The study, published in Science, analyzed monitoring data from 407 cocoa farms taking part in a tree-planting program. The researchers found that auditors were much more likely to change their records when they knew the target a farmer needed to meet to qualify for a reward or pass an assessment. 

In cases where the target was not met initially, subsequent changes were made for 33% of farmers, compared with just 6–7% when the target had already been met. 

The researchers also found a surprisingly simple way to reduce the problem: keeping auditors in the dark about the criteria used to judge whether a farm had passed. 

When auditors did not know the required target, the rate of manipulated data fell from 25% to 11%. 

The findings point to a broader problem for sustainability schemes because the data used to verify environmental claims may not be as independent or reliable as the systems built around it assume. 

False reporting undermines integrity

Governments and companies increasingly rely on farm-level monitoring to demonstrate that commodities meet sustainability standards and to enforce new environmental rules.

Federico Cammelli, a postdoctoral researcher at ETH Zurich’s Environmental Policy Lab, tells Food Ingredients First that the research raises serious questions about the reliability of sustainability claims. Companies may be relying on data that appears to have been independently verified but, in fact, may have been altered before reaching them.The study found that auditors were more likely to alter records when they knew the target farmers needed to meet to qualify for a reward or pass an assessment.

He says manipulation of data can happen due to individual agents’ discretionary power to enter false information, and it goes undetected because of structural conflicts of interests and poor transparency about whether current verification systems are sufficient to counteract incentives to false report.

“It is serious because a final record can look properly verified even when the underlying information has been strategically changed. Companies could therefore rely on data that overstates compliance, despite that data having passed through a formal verification process,” he says.

“At the same time, our study examined one monitoring system in one country. It does not tell us how common the problem is across all sustainability audits, and it does not show that any particular company’s public claims are false. What it does reveal is an important vulnerability that is often overlooked, with a narrow focus on traceability, and too little attention to the credibility of the underlying data.” 

“Companies need to understand who collected the data, what incentives they faced, whether the records were changed, and how the results were independently checked. Sustainability claims would be much more credible if this information were available to the public, including data about the proportion of failed vs passed audits within a supply chain.” 

The research also highlights a less visible driver of false reporting. The pressure to accommodate farmers facing unrealistic sustainability expectations, particularly in smallholder supply chains, is a crucial point, and tackling the problem therefore requires greater engagement with producers, rather than placing the burden solely on reporting systems, says Cammelli.

Conflict of interest in cocoa auditing

Since the findings revealed that changes were much more likely when a farm initially failed to meet the required target — and that auditors specifically altered the data relevant to passing the audit — Cammelli believes that this shows that several actors “can have an interest in a favorable result.” 

“In our experiment, incentives to false reports were minimized: farmers could receive a prize, while agents might want to maintain good relations with farmers, avoid reputational costs, or potentially share the prize. In real sustainability programs, the stakes are substantially higher. Farmers can depend on premiums or market access, while cooperatives can benefit from certified volumes and continued participation in company programs.”

“Traders and manufacturers also value stable supply and evidence that their sustainability programs are working. By design, we excluded these influences, but the broader concern remains structural. When the people producing compliance data are closely connected to those who benefit from compliance, the system creates a conflict of interest.” 

Finding the right solution will require creativity, says Cammelli. But without transparency around data collection and confidence in the credibility of the resulting data, there is little incentive to develop or implement effective solutions.

“Also, part of false reporting may be taking place to provide sufficient slack for poor farmers bearing unrealistic expectations. In that sense, addressing false reporting in full may require substantial engagement of producers, particularly in smallholder supply chains.”