Why Albert Heijn wasn’t the most sustainable Dutch supermarket after all

Opinion piece by Durk Bosma (March 7, 2023), previously published by Marketingfacts.

Soon it will be that time again. On March 23, the Swedish research agency Sustainable Brand Index (SBI) will present the results of its annual survey of how sustainable consumers think brands in 8 European countries are. In every country there will probably be a few glowing CEOs sitting at the table to hear that they have won (again). But should they be so happy?

There has been strong criticism on the study for some years. See also this broadcast of the television program Propaganda, which wondered why Albert Heijn was again the winner among the supermarkets. Last year, the Reclame Code Commissie (the organization that deals with the self-regulation system of advertising in The Netherlands) made an important ruling (also about Albert Heijn) that calls into question SBI’s revenue model.

If SBI has not made fundamental improvements to its research by 2023, all elected winners who communicate about their win may be rapped over the knuckles by the Reclame Code Commissie.

The ruling is not just about Albert Heijn and Sustainable Brand Index. The implications are much larger. After all, lists of the most sustainable, the most customer-friendly, etc., are often published. The attention such an election generates is of course very pleasant for the winners and for the agency that publishes the list. But at the same time not so pleasant for those who are not on the list. And especially not for the consumers who make their choices based on such a list.

Because anyone with a modicum of understanding of research methodology often marvels at the poor quality of research on the basis of which such lists are produced. But SBI is messing up big time. The criticism of their methodology at a glance:

• The perception of sustainabity of a brand is determined on the basis of just 2 questions. In addition, the answers ‘good’ and ‘very good’ are lumped together. A razor-thin score.
• The questions are (also) asked to people who only know a brand by name (and thus cannot say anything about how sustainable the brand is).
• A winner is declared even though there is no statistically significant difference from the runner up. In other words, we are not even sure if the winners are winners. Proclaiming a winner generates more publicity than declaring a shared winner. But this is not allowed by the rules of proper research.
• The selection of brands included in the survey is based on vague criteria. This means that there may be arbitrariness in the choice (or rather, there is room for strategic choices). Within the Dutch supermarket category, the fully organic supermarket chain Ekoplaza, was not in the list so far. Let it be clear, which brands participate ultimately determines who wins. It is also clear that a small brand that is very sustainable can never become the most sustainable brand because of this selection process.
• It is not immediately clear to consumers which brands in a particular sector are and are not included. While consumers may assume that at least all relevant brands have been included. In the case of supermarkets, one might expect Ekoplaza to be included, which could be misleading if Albert Heijn claims to be found the most sustainable.

The new guidelines issued by SBB on communicating about “winning” do not seem to address the above criticisms. The guidelines are mainly about  communicating that the rankings are based on consumer perception (and not actual sustainability).

A complaint to the Reclame Code Commissie last year resulted in a reprimand for Albert Heijn*. Among other things, the RCC stated the following: ‘When an advertiser uses a title as “winner” in communication or campaign, the advertiser has the responsibility, in the context of professional diligence, to make sure that such an outcome and the underlying research are correct. In the opinion of the Committee, this applies all the more if, as in this case, it concerns research into a topical subject, sustainability, and the result, which is favorable to the advertiser, is subsequently used in advertising that directly or indirectly refers to the advertiser’s contribution to maintaining and promoting a clean and safe environment in general. Strict requirements are imposed on such advertising in terms of demonstrability’.

In other words, Albert Heijn should have looked more closely into the quality of the research conducted and questioned whether the research was good enough. Especially since it was a sustainability claim.

Useful rankings that are good

Rankings are useful because they can get things moving within organizations. But then those rankings must be based on solid and transparent research. And all relevant competitors must be included. Moreover, the way results are presented must do justice to the limitations of survey research. Anyone who does not want to take these into account is being deceitful.

And so this is my plea, addressed to all marketers working for a brand that emerges as a “winner” in such a ranking. Before you enthusiastically start typing up a press release, sit down with a research expert to judge the quality of the research behind the win. In many cases, unfortunately, you will find that the research is inadequate. If you fail such an audit, you can expect a rude awakening as happened at Albert Heijn.

*The ruling is only based on the argument that the difference between the No. 1 and No. 2 supermarkets, was too small and did not prove a statistically significant difference. The other arguments were not even considered in the RCC’s ruling.