Micha Tielemans (1792593) – State of the Art Technology – Project Diary (Resit)

PREMISE

“Creating an AR experience visualizing air pollution and natural air purification for the purpose of creating awareness.”

SYNOPSIS

Air pollution is a problem the public has heard about but often can’t seem to grasp. They are aware of the basic principles and effects but don’t know how polluted the air is that they breathe. This is worrying, since generating awareness about air quality could help with pollution reduction (Ramírez et al., 2019). Augmented reality (AR) could help with achieving this goal, by visualizing air pollution. I aimed to learn and use this technology and create an experience to increase awareness about air pollution.

SUBSTANTIATION

Air pollution is a mix of particles and gases, which are known for causing cardiovascular and respiratory problems and environmental damage (Mahajan et al., 2020). Air pollution has an effect on human health, vegetation, materials, the atmosphere, and many more (Boubel et al., 2013). The public is generally satisfied with their current air quality but is feeling somewhat anxious about air pollution. This is very worrying since it has become the leading global risk for public health (Liu et al., 2016).

Augmented reality (AR) technology uses the physical world and displays an “augmented” reality by adding virtual computer-generated information to add to the interaction with the real world (Carmigniani et al., 2011). Studies have shown that users had an increase in conceptual knowledge, topic interest, engagement, and awareness after using AR for informal Science learning interactions (Goff et al., 2018).

There have been instances where AR increased awareness about environmental issues. There has been a project, where Air quality data was modeled and augmented through AR miniature trees in the immediate surroundings. This helps with a better understanding of the local context (Prophet et al., 2018). The use of AR with the integration of gamification concepts has helped to improve environmental knowledge and awareness (Mei & Yang, 2019).

PROJECT OVERVIEW

I want to create an AR experience that makes air pollution “visible” and shows how vegetation contributes to air purification. The phone will show the real world, where the air is filled with particles. When the user is looking at a certain particle, information will be displayed about what it is, what it contributes to air pollution, and where it originates from. Users can also look at vegetation and receive information about how it purifies the air.

Prototype 1: Proof of concept (Artivive)

I decided to create a proof of concept, to investigate if my idea was accomplishable in the online program Artivive.

Actions:

• I gathered general information about the air purification qualities of one of my house plants (Walden, 2021).
• Generate a video containing this information to use as an overlay in Artivive.
• Create 2 models with different images to be recognized: A stock photo and a photo of my physical houseplant.

Insights:

• Proof of concept generated an experience that I envisioned.
• The prototype was able to scan real-life objects with the use of 2D pictures.

Figure 1 Artivive Model

Figure 2 Proof of concept: Picture     

Figure 3 Proof of concept: Real life

Prototype 2: CO₂ molecule (Unity)

During my research into existing AR examples about air pollution, I found a video (AR Critic, 2019) that corresponded with my vision. But I want to display information about each particle.

I decided to use Unity, a real-time development platform for games, VR, and AR, for the rest of my project. With AR Foundations, it is possible to use the ARCore XR Plugin on android to run prototypes on my mobile phone. Unity uses C# coding scripts to add functionality to prototypes.

Actions:

• Used a tutorial (Third Aurora, 2020) as a concept to display my information.
• Modeled a CO₂
• Gathered basic information about carbon dioxide (Somma, M., 2021), that could be displayed.
• Tested the APK on my Phone.

Insights:

• How to use Unity.
• How to write in C#.
• How the model works in the APK on my phone.
• The prototype functions as intended.

Figure 4 Unity CO2 model

 

Figure 5 C# code for Pop-up animation information

 

Figure 6 C# code for activating Pop-up animation when looked at with camera

 

Figure 7 C# code for rotating information with camera view

 

Figure 8 Video CO2 model

 

Prototype 3: Multiple particles (Unity)

My next iteration was about adding multiple particles. I decided to focus on the current pollutants of interest. I used the foundation of the CO2 model from the previous iteration and decided what information I wanted to display.

Actions:

• Gathered additional data about Ozone (US EPA, 2016a), No2 (US EPA, 2016c), Particle matter (US EPA, 2016b), PM2.5 (Fine Particles (PM 2.5) Questions and Answers, n.d.) and combined them with information from my literature review (Brunekreef & Holgate, 2002).
• Collected that data from a Dutch air quality index (Utrecht – de Jongweg Air Quality Index (AQI) and Utrecht Air Pollution | IQAir, n.d.) and picked the data from the nearest place near campus.
• Model the other polluting particles.

Insights:

• 5 particles were sometimes barely visible, so I enlarged the PM2.5 spheres and the PM10 accordingly.
• The capsule of the information board for the PM2.5 particles was too large.
• The starting position of the models was too far to the right. I corrected the beginning position of the AR camera.
• You had to look too far down to interact with the models. I raised the positions of all the models to be above Y=1.
• Added light estimation to models, to enhance the feel of immersion of virtual aspects into the real world.

Figure 9 Video Prototype 3, first iteration

 

Figure 10 Unity Prototype 3, second iteration

 

Figure 11 Video Prototype 3, second iteration

 

I send out a survey, containing a video of the latest prototype.

Insights from peer feedback:

• Use shorter/summarized information text.
• Make text more visible.
• Enlarge particles.
• What the design represents is clear.
• Way of interaction is nice. Would like to pause the interaction so it doesn’t disappear quickly.
• Made them more aware about air pollution.
• Add if the air pollution in your nearby surroundings is “Okay’ or “Bad”.

Figure 12 Survey prototype 3

 

After I reviewed the answers to the survey, I concluded that it was rather difficult to get some insight into how users would physically interact with the prototype.  I did a small interaction test with my prototype with 3 participants.

Insights from user testing:

• Participants found the interaction intuitive and understood how to get the information to pop up.
• It was difficult for the users to keep the information visible and not let the pop-up close. This was probably due to the information being mostly displayed on the bottom of the screen. Naturally, users want to center it and look too far down, and the AR camera couldn’t trigger de animation anymore.

 

Future prototype

Fixes for the next iteration(s):

• Make the information board not transparent, so information is more readable.
• Summarize effects of particles for easier viewing.
• Remove the information about maximum and current concentration. Use a color code to indicate if a value is “good” or “bad”.
• Use an invisible object to activate the pop-up animation. This way It could be bigger and make it easier for users to keep reading the information.

 

Conclusion

The AR experience is a helpful and new tool to create more awareness about air pollution. It still needs to have a comprehensive research phase, including how users will continue to use the prototype and if they will use their newfound insight in any meaningful way. During this project, I managed to learn a lot about the technology and the possibilities it could provide.

AR has a lot of possibilities to expand this prototype and take it even more tailored to its user. We could include Plane Occlusion (Anton Developer, 2020). This way the virtual particles will get obstructed by real-life objects and feel more incorporated into the environment. Including Realtime data (Third Aurora, 2021b) could give a data-driven element to the prototype. And we could add location-based (Unity AR GPS Location, 2020) data from air quality monitors. This way we could generate an AR experience that matches the close environment of the user. Another way of creating a location-based experience is by using Vuforia Area targets (Unity, 2020). This way you could scan your desired environment and use Unity to recognize the scanned environment and display a tailormade experience.

During this project, the part about users interacting with vegetation and receiving information about how it purifies air has been absent from later prototypes. But this functionality could be added to future iterations. We could implement the technique of image tracking (Dev Enabled, 2020) and use the same principles as my Artivive prototype. More advanced techniques like object recognition (Unity, 2019) or use machine learning (Third Aurora, 2021a) could also be used for the recognition of different plants and trees.

REFERENCES

Anton Developer. (2020, June 17). Unity ARFoundation Plane Occlusion | Tutorial (Android/IOS). https://www.youtube.com/watch?v=XxPT2WTrPoU

AR Critic. (2019, February 22). Air App Review—Visualize Air Quality in AR. https://www.youtube.com/watch?v=o_QkgYLqefY

Boubel, R. W., Vallero, D., Fox, D. L., Turner, B., & Stern, A. C. (2013). Fundamentals of Air Pollution. Elsevier.

Brunekreef, B., & Holgate, S. T. (2002). Air pollution and health. The Lancet, 360(9341), 1233–1242. https://doi.org/10.1016/S0140-6736(02)11274-8

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51(1), 341–377. https://doi.org/10.1007/s11042-010-0660-6

Dev Enabled. (2020, April 5). AR Foundation Improved Image Tracking—Multiple Objects/Images—Unity Augmented Reality/AR. https://www.youtube.com/watch?v=I9j3MD7gS5Y

Fine Particles (PM 2.5) Questions and Answers. (n.d.). Retrieved April 5, 2022, from https://www.health.ny.gov/environmental/indoors/air/pmq_a.htm

Goff, E. E., Mulvey, K. L., Irvin, M. J., & Hartstone-Rose, A. (2018). Applications of Augmented Reality in Informal Science Learning Sites: A Review. Journal of Science Education and Technology, 27(5), 433–447. https://doi.org/10.1007/s10956-018-9734-4

Liu, X., Zhu, H., Hu, Y., Feng, S., Chu, Y., Wu, Y., Wang, C., Zhang, Y., Yuan, Z., & Lu, Y. (2016). Public’s Health Risk Awareness on Urban Air Pollution in Chinese Megacities: The Cases of Shanghai, Wuhan and Nanchang. International Journal of Environmental Research and Public Health, 13(9), 845. https://doi.org/10.3390/ijerph13090845

Mahajan, S., Kumar, P., Pinto, J. A., Riccetti, A., Schaaf, K., Camprodon, G., Smári, V., Passani, A., & Forino, G. (2020). A citizen science approach for enhancing public understanding of air pollution. Sustainable Cities and Society, 52, 101800. https://doi.org/10.1016/j.scs.2019.101800

Mei, B., & Yang, S. (2019). Nurturing Environmental Education at the Tertiary Education Level in China: Can Mobile Augmented Reality and Gamification Help? Sustainability, 11(16), 4292. https://doi.org/10.3390/su11164292

Prophet, J., Kow, Y. M., & Hurry, M. (2018). Cultivating Environmental Awareness: Modeling Air Quality Data via Augmented Reality Miniature Trees. In D. D. Schmorrow & C. M. Fidopiastis (Eds.), Augmented Cognition: Intelligent Technologies (Vol. 10915, pp. 406–424). Springer International Publishing. https://doi.org/10.1007/978-3-319-91470-1_33

Ramírez, A. S., Ramondt, S., Van Bogart, K., & Perez-Zuniga, R. (2019). Public Awareness of Air Pollution and Health Threats: Challenges and Opportunities for Communication Strategies To Improve Environmental Health Literacy. Journal of Health Communication, 24(1), 75–83. https://doi.org/10.1080/10810730.2019.1574320

Somma, M. (2021, October 20). The Effects of Carbon Dioxide on Air Pollution. Sciencing. https://sciencing.com/list-5921485-effects-carbon-dioxide-air-pollution.html

Third Aurora. (2020, April 21). Augmented Reality Tutorial | Gaze Interaction in Unity. https://www.youtube.com/watch?v=OE66gtiF8QQ

Third Aurora. (2021a, February 9). Machine Learning Models in Unity with Barracuda: Image Classification. https://www.youtube.com/watch?v=LhzKfx2kuDs

Third Aurora. (2021b, July 13). Realtime Unity Data with Google Sheets | Third Aurora Augmented Reality Tech Company. https://www.youtube.com/watch?v=zgSES5PQNu8

Unity. (2019). AI in Unity: Recognize objects in your AR applications – Unite Copenhagen. https://www.youtube.com/watch?v=s5cBtsIS3_A

Unity. (2020, November 19). Spatial augmented reality with Vuforia Engine in Unity | Unite Now 2020. https://www.youtube.com/watch?v=lomyM4loAVg

Unity AR GPS Location. (2020, January 31). Unity AR+GPS Location—Web Editor. https://www.youtube.com/watch?v=7-WCEVuSkK8

US EPA, O. (2016a, March 21). Health Effects of Ozone in the General Population [Data and Tools]. https://www.epa.gov/ozone-pollution-and-your-patients-health/health-effects-ozone-general-population

US EPA, O. (2016b, April 19). Particulate Matter (PM) Basics [Overviews and Factsheets]. https://www.epa.gov/pm-pollution/particulate-matter-pm-basics

US EPA, O. (2016c, July 6). Basic Information about NO2 [Overviews and Factsheets]. https://www.epa.gov/no2-pollution/basic-information-about-no2

Utrecht—De Jongweg Air Quality Index (AQI) and Utrecht Air Pollution | IQAir. (n.d.). Retrieved April 5, 2022, from https://www.iqair.com/netherlands/utrecht/utrecht-c/utrecht-de-jongweg

Walden, S. P. + L. (2021, October 12). 26 best air purifying plants for the home. Country Living. http://www.countryliving.co.uk/homes-interiors/a668/houseplants-to-purify-house-air/