“Sustainability is now a big baggy sack in which people throw all kinds of old ideas, hot air and dodgy activities in order to be able to greenwash their products and feel good.” – Kevin McCloud

Permise

Detecting greenwashing with deep learning and supporting sustainable consumers in buying actual sustainable products.

Synopsis

In this project, I conduct experiments to discover whether deep learning is capable of detecting greenwashing. After putting more research into certain brands, I discovered I fell for multiple greenwash practices. I decided to interview other sustainable consumers, and I discovered I’m not alone. Almost everyone I spoke to admitted that he/she fell for greenwash marketing practices once or even multiple times. This showed me the scale of the greenwash problem. The result of this project shows a concept diary consisting of a greenwash research, technological experiments, and visualisation through an app.

My own greenwash collection!

Insights

The insight I hope to retrieve from this project:

• Choosing or creating the right dataset for the prototype;
• Getting a better understanding of neural networks;
• Employing different kind of detection and recognition technologies.

Preface

1 Greenwashing factors

Greenwashing refers to being visually sustainable instead of actually sustainable. In this case, companies portraying their product as environmentally friendly and hope to persuade potential sustainable consumers in buying their “green” brand (Faizal, 2021). 66% of consumers are willing to do their part in buying sustainable products (The Nielsen Company, 2021). Buying sustainable products may seem like an accessible act to contribute to a sustainable world without putting too much effort into it. Checking the actual greenness of a company is the responsibility of the consumer.

Which packaging properties are visible for the consumer?

Adformate (2021) explains consumers switch between two states while shopping. In the first state the consumer looks at the simple aspects of the packaging such as color. The moment they find an interesting brand they will look more precisely at the logo and text properties.  

This project focusses on “green” consumers. Green consumers are prepared to go one step further in finding actual sustainable products. In this case, we can say the consumers only shops in state two, and are focussed on sustainable sensory aspects.

Sustainable sensory aspects

Nemat et al. (2019) describe sensory aspects of packaging that can communicate with the consumer. Extrinsic sensory aspects factors are, text, pictures, icons, symbols and colours. All these factors create a green perception for the consumer.

Greenwash sensory aspects

Factors that are frequently used  to create a green perception are:

• Vague buzzwords
• Idyllic imagery
• Hidden parent companies
• Lack of transparency
• Counterintuitive values

AI for greenwashing 

According to Cojoianu et al. (2020) Ai can be an effective tool in terms of detecting misleading packaging. The aim of their tool is to analyse the claims of companies worldwide and test them with the actual “greenness” of their company.

Why text?

To test which factor are most visible for the consumers, I first did a supermarket research. By taking pictures of green appearing products I can determine which factors occur the most on the green products.

By scanning through the images I can concluded the green perception mostly arises from buzzwords and imagery. As a second step, I conducted small interviews to check the outcome of my first research with potential green consumers.

Findings:

• Green consumers firstly look to the imagery and then decide whether they buy the product by reading text (potential buzzwords);
• and green consumers stick to buying one products from one particular “green” brand.

Based on these outcomes I decided to focus on the green buzzwords. Also, buzzwords are convenient to measure for the consumer, because they are placed on the front of the packaging.

Preparations

2 Optical character recognition

In AI terms packaging text is unstructured text which can be recognised by deep learning OCR (optical character recognition). The aim of my prototype is to convert buzzwords into an alert for the green consumers. I can do this by detecting text with efficient accurate scene text detection (EAST) or regions of interest (rio), and recognising the text with a convolutional neural network (CNN) or Pytesseract.  All elements are employed with OpenCV.

Buzzword selection

Companies use buzzwords such as, 100%, all natural, bio, biodegradable, biological, cruelty free, eco friendly, environmentally friendly, ethical, free from, natural, no chemical, non toxic, organic, planet, plastic free, recyclable, sustainable, vegan etc… (The zero waster, 2021).  For my prototype I’m focussing on these words.

3 Creating a dataset

The datasets are employed in different stages of the prototype.

Attempt 1 

For the first dataset I tried to find text in the wild on packaging products. I created folders to cover each buzzword and took screenshots from packaging text images in combination with screenshots from Google font images.

 

Attempt 2 

For the second dataset I focussed only on collecting structured text from Google font.

Attempt 3

The third dataset consist out of structured Google font images converted into the same dimension.

Prototyping

4 Technology 1

Applying text detection with OpenCV

EAST text detector is a deep learning model that can create bounding boxes around the buzzwords. By using OpenCV for video capturing, image optimising and image employment, EAST can be used for creating boxes around different text areas in frozen images and real-time videos. After retrieving an image or video through my webcam I attempted to create boxes around the text.

Attempt 1

Attempt 2

Attempt 3

 

[aesop_video src=”youtube” id=”4TGhMExjcmg” align=”center” disable_for_mobile=”on” loop=”on” controls=”on” mute=”on” autoplay=”on” viewstart=”off” viewend=”off” show_subtitles=”off” revealfx=”off” overlay_revealfx=”off”]

 

 

 

Learning: the first two attempts show that the bounding boxes don’t cover the whole words and are very fuzzy. This provides a bad input for the CNN model. By testing the EAST model with more structured text, I can see whether the model is capable of detecting every part of the text.

Testing the EAST model with structured and unstructured text

Attempt 1

Attempt 2

Attempt 3

Learning: the model is not capable of detecting unstructured tex.

5 Technology 2

Training the CNN model with Tensforflow Keras.

Attempt 1

Training and testing the model with dataset 1.

63,22 % validation accuracy

Attempt 2

Training and testing the model with dataset 2.

98,49 % validation accuracy

Attempt 3

Training and testing the model with dataset 3.

98,49 % validation accuracy

Learning: the model reaches the highest validation accuracy with dataset 2 + 3. There is no different in accuracy by changing the dimensions of the input images.

6 Prototype 1

Testing the CNN model with OpenCV real-time video capturing.

Attempt 1

Testing with the model trained on dataset 1 on unstructured wild text.

Attempt 2

Testing with the model trained on dataset 1 on unstructured wild text. and building in a threshold. If the model is 65% sure about its prediction, it will show the recognised buzzword.

Learning: the labels are not well connected to the different buzzwords or the bounding boxes are not precise enough.

Attempt 2

Testing with the model trained on dataset 3 on structured text.

Learning: the model its favourite buzzwords is biodegradable and the model is not trained on the right dataset.

7 Iteration 1

Applying text detection with rio, hsv (hue, saturation, value) image optimising, and OpenCV images capturing. By changing the hsv of an image I can create a rio.

Attempt 1

(hMin = 0 , sMin = 0, vMin = 171), (hMax = 179 , sMax = 57, vMax = 223)

Attempt 2

Text in the wild.

(hMin = 81 , sMin = 6, vMin = 176), (hMax = 110 , sMax = 130, vMax = 255)

Learning: rio can’t be created by changing the hsv of unstructured text in the wild.

Attempt 3

Focus on the unstructured text.

(hMin = 0 , sMin = 0, vMin = 188), (hMax = 23 , sMax = 69, vMax = 255)

Focus on flat structured text.

(hMin = 11 , sMin = 21, vMin = 0), (hMax = 94 , sMax = 49, vMax = 255)

Learning: model can both detect unstructured and structures flat text. However, it can’t detect text with different colours.

Testing the CNN model with OpenCV on rio on a frozen image.

Attempt 1

Learning: the EAST model can detect the text, but the CNN model can’t recognise the word.

8 Prototype 2

Text detection with EAST and recognition with a pre-trained Pytesseract OCR model. 

Attempt 1

Learning: Pytesseract can recognise and output words. It’s still difficult to detect different text boxes on one packaging label.

9 Implementation

Design

By visualising the input and output of the model, the app provides a consumer-friendly alert. Also, it leaves open the decision for the consumer to gain more knowledge about a certain product or consult a greenwash expert.

Try it for yourself:

Note this is not a working prototype some parts are not clickable!

https://xd.adobe.com/view/2acf662e-4fb4-4884-83ad-3c98b3f949d7-7f9c/?fullscreen&hints=off

Feedback from sustainable consumers

” The app can connect with actual green brands. In this case, the app can point out the green brand as a replacement for the greenwashed product.” – Chiem

” Clarify the connection between the consumer and the system. Especially on the fourth screen”. – Ilse

“This app can provide me the knowledge that I need to make the right decision.” – Marthe

” The app provides a clear overview of the input that I need to provide and the output that I can use to make the right decision in buying products.” – Tommie

“With this app I can buy sustainable products without putting too much effort in it. ” – Eva

Reflection

For me, the employment of text detection and text recognition was a process of trial and error. The process of exploring different image sources and retrieving images was almost a project on its own. By watching youtube videos, tutorials, and reading articles I managed to learn a lot about Tensforflow Keras, OpenCV, EAST, CNN, en Pytesseract. With this knowledge, I was able to create bounding boxes in an image, create and train a CNN model, implement a pre-trained model, use the output from one model as input for a second model, retrieve input through my webcam, and run a code with a real-time and frozen image input. Unfortunately, I wasn’t able to make a fully functional prototype. Therefore, I could only test the prototype by myself and explain and present my idea to fellow students and friends. Besides the unfinished prototype gaining knowledge was the most rewarding. It was a lot of fun to work on an experimental process as I did in my previous study Product design.

Conclusion

Exploring computer vision was a great adventure. My process shows that computer vision can serve consumers in taking over their literacy and increase their knowledge about greenwashing. In the end, my prototype didn’t work out according to my plan. Therefore, I can conclude detection and recognising unstructured text on packaging is challenging but not impossible. To test if sustainable consumers are interested in my prototype, I visualised the input and output of the prototype in an app. The outcome of my experiment and the feedback of sustainable respondents show that my initial idea for the prototype has great potential for consumers. In this way, consumers can feel confident about their product and brand choice.

[aesop_video src=”youtube” id=”uY4q0L2YOe4″ align=”center” disable_for_mobile=”on” loop=”on” controls=”on” mute=”on” autoplay=”on” viewstart=”off” viewend=”off” show_subtitles=”off” revealfx=”off” overlay_revealfx=”off”]

References

Adformate. (2006). Hoe ziet een consument een product in het schap? Adformatie. https://www.adformatie.nl/contentmarketing/hoe-ziet-een-consument-een-product-het-schap#:%7E:text=Consumenten%20zijn%20geneigd%20producten%20in,op%20ooghoogte%20kunnen%20worden%20uitgestald.&text=Van%20der%20Lans’%20onderzoek%20laat,razendsnel%20wisselen%20tussen%20twee%20staten.

Cojoianu, T., Hoepner G. F. A., Ifrim, G. & Lin, Y. (2020). Greenwatch-shing: Using AI to detect greenwashing. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3627157

Faizal, F. (2021, January 21). What Is Greenwashing? – Types & Examples. Feedough. https://www.feedough.com/what-is-greenwashing-types-examples/

Nemat, B. N., Razzaghi, M., Bolton, K., & Rousta, K. (2019). Attributes of food packaging that influence consumer behavior (including recycling behavior). [Figure]. https://www.mdpi.com/2071-1050/11/16/4350

The Nielsen Company. (2021). The global, socially-conscious consumer. https://www.nielsen.com/wp-content/uploads/sites/3/2019/04/Nielsen-Global-Corporate-Social-Responsibility-Report-June-2014.pdf

The Zero Waster. (2019, January 13). Glossary of Greenwashing. https://thezerowaster.com/glossary-of-greenwashing/