Counting plastic bottles on beaches with computer vision and drones

premise Using computer vision technology and drones, a system was developed to automate standing stock surveys of plastic waste. synopsis I investigated an alternative method for surveying standing stock of plastic litter on beaches. Traditionally, surveyors walk methodically around beach sections, manually recording all plastic litter encountered (Opfer et al., 2012). In comparison, I propose […]

Read more "Counting plastic bottles on beaches with computer vision and drones"

Using IoT devices to influence privacy behavior

Premise Using informative IoT devices and gamification mechanisms to influence privacy behavior. Synopsis Research suggests that many people are aware of online privacy problems. However, this awareness does not seem to translate into action. People for example still choose to use privacy-unfriendly services and oftentimes provide more data than necessary. This discrepancy is described as […]

Read more "Using IoT devices to influence privacy behavior"

Developing an Adaptive, Assistive Environment Awareness tool for the Deaf and Hard of Hearing Group

Premise Using acoustic event detection and convoluted neural networks to develop an optionally isolated,  adaptive,  assistive environment awareness system for the deaf and hard of hearing group utilizing haptic notifications on a smartphone and/or smartwatch. Synopsis Sound occurring in our environment gives us details that provide us with situational awareness. Individuals who have auditory impairment […]

Read more "Developing an Adaptive, Assistive Environment Awareness tool for the Deaf and Hard of Hearing Group"

Safe sexting; the functionality of smart contracts within sharing explicit photos

Sophie van Dael                                        1713547                              State of the Art

Premise

Exploring and explaining the functionality of smart contracts in the protection of explicit photos shared through sexting among 18 – 35-years-olds.

Synopsis

As all aspects of modern human life evolve to include online components, it is only common sense that dating transformed as well. As dating shifts to an online environment, new risks arise. A huge risk affecting those who experience (part of) their relationships online, has been the ease with which explicit photographs can be shared to others. This practice is commonly known as ‘revenge porn’.  This occurs when a partner shares intimate photos send to them in private to others. To shift the conversation and provide users with a proactive stance towards this issue rather than a restrictive one, I will be researching the functionalities of smart contracts and to what extent they can aid in combatting this issue.

Revenge porn

Revenge porn can be defined as the distribution of sexually explicit photographs to third parties with the intent to harm, or gain control of the victim (Harper et al, 2021). Over the past few years, countries have undertaken steps to set-up a legal framework to address this offense, but actionable solutions to prevent the crime from happening have yet to be proposed (Ruvalcaba, 2021). Instead, most solutions are aimed at restricting the sender, and often protect the perpetrator through a sense of invisibility (Scott et al., 2018).

Although sending explicit photos bears consequences, it does not stop people from sending explicit pictures to their partner: research reports about a third to one half of adults between the ages of 18 to 26 engage in this behavior (Benotsch et al., 2012). This is most commonly done through online acquaintance based environments (ABE), where users have relationship together (Jorgensen &  Demant, 2021). As this form of intimacy displayed by new generations is obviously desired, safer alternatives to this should be explored where protection of the sender takes center stage.

Smart contracts

Smart contracts are an important beneficial component of blockchain technology. They are self-executing contracts, in which the terms and conditions are set using software codes (Buterin et al., 2014).  The contracts cannot be forged, and data protection is at the center of this technology, they also allow for vigilant identification of users and the right to revoke access to certain data (Zhang et al., 2021).

After initial research into the technical side of smart contacts, the best way to explore and create an entry-level smart contract seems to be the program language Solidity. Solidity is a new language launched to function on the Ethereum blockchain. It bears similarities to languages such as Python, and functioned as a good starting point for me to discover the technical side of smart contracts. As participants were not knowledgeable about programming, I translated the functions of the smart contract in my prototype to a visual design, for users could not provide insight into programming.

Simple smart contract in solidity; protecting data variables.

Prototyping

To research if the introduction of a smart contract into the online exchange of explicit photos is meaningful to users, I asked 4 people who have previously shared explicit photos to be the test audience for my iteration rounds.

Iteration

Iteration 1

Prototype

As none of the participants had any experience with smart contracts, my first iteration focused on explaining the workings in the context of this problem. To ensure the understanding of the participants, I created a low-fidelity prototype through paper and attributes, such as keys, a paper contract, and a Tupperware.

  1. First iteration round

 

Testing

I tested the prototype per duo, and asked them through role-play to imagine being in the scenario of sending a photograph. I used Tupperware to resemble the safe locking of data on the blockchain. Both were given keys, which with they could access the photograph. Through storytelling, I explained the main functions the smart contracts introduces (the security layer, individual ‘keys’, and the right to revoke access by one of the partners).

Results

The very simplistic prototype ensured the participants would not be overwhelmed while clearly explaining the attributes of smart contracts. The storytelling element was essential, as it conveyed to the participants an understanding of the severity of being a victim to this crime, and the lack of self-protection. Benefits of the prototype included the right to revoke access, although the participants were unclear how this would work into real-life online situations.

Insights

  • The risk of sending explicit photos is noted but not feared to a level that results in abstaining.

Solutions focused on restriction are futile.

  • The trust for the partner, combined with a sense of duty, trumps the potential consequences.

The solution needs to be easily integrated into their habits, too much effort will result in non-use.

  • A proactive need for a way to protect the sender is necessary, as the trust alone is not enough long-term.

Protection of the senders should be prioritized.

Iteration 2

Prototype

The main question left unanswered by the participants was how the smart contract would function in real life. To display this, I created a paper prototype displaying the attributes of the smart contract. These are:

  • Both parties can only access the photo with a mutual ‘key’, which they can use.
  • When the relationship has ended, one can revoke access.

2. Attributes of the smart contract, paper prototype

Testing

The second testing phase focused on the selection of the parameters and attributes of the smart contract. Through co-creation, participants were encouraged to critique and add to said smart contract. To implement the smart contract into their daily lives, the prototype showed a texting environment similar to WhatsApp.

  1. Co-creation, suggestion of alternated attribute by participant.

Results

All participants valued the prototype being integrated into a familiar environment. Getting an access ‘key’ to see the photograph stimulated a sense of safety and was important to all participants. One participant remarked the importance of not limiting revoked access to only the end of the relationship, but rather be an option to the sender at all times (in event of a fight, cheating etc.)

Insights

  • The right to revoke access to shared photos should be more flexible, the sender should be able to set the parameters.

Access should be broadened so access to photos can be revoked at any given moment.

  • The participants value the smart contract and its attributes, but they are not likely to seek this out alone.

The smart contract should be employed within a familiar environment.

  • Participants suggested the ‘no screenshot’ function of Netflix as extra protection.

Photos cannot be screenshotted.

Iteration 3

Prototype

4. Attribute of online prototype; the red button allows the sender to restrict access.

To implement the changes into a visual, online design, all feedback was realized into an updated online prototype in Figma, that displays the working of the smart contract.

Testing

Participants were asked to interact with the prototype online and judge the overall concept according to a user-experience questionnaire.

Results

The participants appreciated the implementation of the smart contract into their familiar surroundings. They acknowledged the change in perspective it brought into the consequences of sending explicit photos and the lack of self-protection currently available. They were pleased with all implemented features.

Insights

  • It is important for participants to be able to retract consent to photos at any given time.

A key value of the final prototype is the autonomy of the sender.

  • The prototype shifts the perspective of participants to better protect themselves from their partners distributing their photos without consent.

The prototype offers insightful information to the possibility of protecting the sender which is currently not provided by traditional online channels, such as WhatsApp.

Further research

For future perspectives, it would be beneficial to investigate the technical possibilities and challenges of creating a plug-in smart contract, and if the Ethereum network is a mere option, or if there are other ways of pursuing this (possibly even outside of blockchain).

References

Benotsch E., Snipes D., Martin A., Bull S. (2013). Sexting, substance use, and sexual risk behavior in young adults. J. Adolesc. Health 52, 307–313.

Harper, C., Fido, D., and Petronzi, D. (2021). Delineating non-consensual sexual image offending: Towards an empirical approach. Retrieved from; https://www.sciencedirect.com/science/article/pii/S135917892100001X

Jørgensen, K. E., & Demant, J. (2021). Shame, shaming and economy: A theory of image-based sexual abuse within different online sharing environments. First Monday, 26(4). https://doi.org/10.5210/fm.v26i4.11670

Ruvalcaba, Y., & Eaton, A. A. (2020). Nonconsensual pornography among U.S. adults: A sexual scripts framework on victimization, perpetration, and health correlates for women and men. Psychology of Violence, 10(1), 68–78. Retrieved from; https://doi.org/10.1037/vio0000233

Scott, G., Brodie, Z., Wilson, et al. (2020) Celebrity abuse on Twitter: The impact of tweet valence, volume of abuse, and dark triad personality factors on victim blaming and perceptions of severity. Retrieved from; https://www.sciencedirect.com/science/article/pii/S0747563219303462?casa_token=pebeQGm_miwAAAAA:SxmupLgJCREi3e_eYmMIaOS4NR8uN5Hn-RXQtU8j5EeudV7vro8ywSfqTSxXIJ02QZqyKi4wxvEB#bib40

Zhang et al., (2021). Secure and Efficient Data Storage and Sharing Scheme Based on Double Blockchain. Retrieved from; https://scholarworks.bwise.kr/sch/handle/2021.sw.sch/2215

Read more "Safe sexting; the functionality of smart contracts within sharing explicit photos"

In The Eyes of The Recruiter

PREMISE Detecting body language during the online interview using deep learning for image classification. SYNOPSIS Recruiting is a more complex activity than most candidates think it is. It does not just involve answering the “right” questions related to the job post, it is about making the recruiter know you better so you can land the […]

Read more "In The Eyes of The Recruiter"

Combining GSR data and music to reduce stress

PREMISE

Exploring ways to create an effective stress management method using GSR data and music

SYNOPSIS

Stress is undoubtedly a pandemic by itself. With so many different stressors that we have no control of the only thing we can do is focus on the things that we have control over – our mind and body. There are many proven techniques to manage stress such as exercise or guided mindfulness practice, however, most of them have their limitations and are not easily accessible to everyone. I wanted to find a way that would not only be highly accessible and fun to use but also be evidence-based. I chose to explore ways how we can use GSR biosensor to measure people’s emotional response to relaxing music and recommend them songs that will further enhance their stress reduction effectiveness.

PREFACE

Nobody is immune to experiencing stress, regardless of age, sex, race etc., however, a survey by the American Psychological Association revealed that Gen Z (15-21-year-olds) and Millennials (22-39-year-olds) in the US reported higher than average stress levels and higher than any other generation (APA, 2018). Similarly, a study in the Netherlands found young people to be the biggest population group (82%) at risk of burnout in the past year, a high increase from the previous year (NCPSB, 2022). While the main reason identified for such spike in numbers was Covid-19 related stressors such as job insecurity, financial problems, school-related issues, affected relationships etc. (Graupensperger, Cadigan, Einberger, & Lee, 2021), these and similar issues have been weighing down young people for years before the pandemic. One of the most mentioned causes of stress in young people in the existing research is academic-related.

When stress is not managed it can wreak havoc on a wide range of areas of an individual’s life. According to Hazen et al. (2011), as a response to stress, young people commonly experience anxiety, moodiness and irritability, as well as cognitive issues such as difficulty concentrating, which has a negative impact on students’ academic performance (OECD, 2017). Over time, unmanaged stress paves the way for more serious mental health issues such as depression (Kessler, 2012). Physical health is also at risk, research has shown that stressed people are less likely to exercise and more likely to overeat (Dallman et al., 1993; Stults-Kolehmainen & Sinha, 2013).

One of the most common ways students attempt to manage stress is by listening to music (54.8%), others watch internet videos, talk to friends and even indulge in substance abuse (Mahani & Panchal, 2019). Although there is no data on how popular stress-management apps are among young people, a study found that roughly a half of them were not evidence-based and likely were not effective anyways (Coulon et al., 2016). Given the current state of stress and the highly detrimental consequences it has on young people, there must be a solution that is easily accessible to everyone, evidence-based and engaging. With the currently available technological affordances such as wearable biosensors and an abundance of stress management research in the past decades, there is an opportunity for creating a data-driven solution for this problem.

PROTOTYPING

Building a song database

To begin building the prototype, I first needed to have a database with a lot of different potentially relaxing songs. To achieve that, I created a Spotify playlist and added 50 songs that I found in other playlists curated by Spotify (i.e. “Calming Classical”, “Deep House Relax”, “lofi beats”, “Stress Relief” etc.). Then, using Spotistats, I was able to export the playlist data with all the song characteristics such as tempo, genre, valence, acousticness, instrumentalness and many more. After some data cleaning with python, this is what it looked like:

Inducing stress

Before I start testing songs on people, I need to make sure they are in a stressed state. After a few unsuccessful attempts to induce stress, I found a method that worked relatively well. It is called the mental arithmetics task. The participant is asked to subtract 7 from 500 and then repeat as fast as possible (500-7, 493-7, 486-7 etc.). Below you can see how well it worked over 30 seconds – the GSR reading went from around 280 to above 300.

 

Iteration 1

Testing the songs and creating a recommender system

 

Arduino Uno board and GSR sensor by Grove

 

Now that I am able to put the participant in a stressed state, I can start measuring stress. To do that, I started by playing a random song while the participant was wearing the GSR electrodes on their fingers. This is an example of the GSR response to one of the songs:

 

As you can see the GSR dropped in the beginning and then started slowly increasing.

 

I repeated the process together with the mental arithmetics task for seven different songs and stored the data in separate JSON files. At this point, I am still not sure what is the best way to make meaning out of all the ups and downs in the GSR data that I collected. Therefore, for the initial version of the recommender system, I simply used the difference between the first (t=0 sec) and last (t=500 sec) GSR values registered, to determine how effective was the song in reducing stress. I added these values ( “before”, “after”, “difference” ) to the data frame for every song I tested on the participant.

Building a recommender system

I now have a data frame with song titles and their features, as well as GSR data indicating how well some of the songs worked on the testing participant. Below you can see that the song called “Hold On” worked the best out of the other few that I tested. It decreased the GSR by 67 units.

Now I want to build a recommender system that will be able to recommend the user songs that are similar to the one that performed the best. This type of recommender system is called content-based filtering. I will be using a cosine similarity machine learning model, which compares the similarity of two feature vectors. The first step is to select the features I want the recommender system to base its recommendation off and transform them into the range between 0 and 1. I did it using MinMaxScaler from sklearn and used the following features: ‘Energy’, ‘Key’, ‘Loudness’, ‘Mode’, ‘Speechiness’, ‘Acousticness’, ‘Instrumentalness’, ‘Liveness’, ‘Valence’, ‘Tempo’. Then using the sklearn’s cosine_similarity library and the code I found and adapted from machinelearninggeek.com, I was able to generate a list of top 10 songs from the database that were the most similar to “Hold On” (the one that performed the best so far).

The recommended songs

 

Iteration 2

Testing the recommendations with an app interface

Then l tested the first couple recommendations to see how they performed. In addition, I created an interface for the music player with the recommender system, which I also tested out with the user. One feature of the interface is seeing your GSR numbers in real-time. I personally like seeing them because it helps me concentrate on relaxing and my breathing, however, I can imagine that for some people it might be stressful to watch them, especially if the numbers are increasing. Unfortunately, I was not able to implement this feature into the design due to my limited time and skills. Therefore, I tested this feature by asking the participant to watch the live numbers on the serial monitor of Arduino IDE on a laptop.

 

The top two recommended songs were “Romeo” and “Mutual Feeling”. The first song performed relatively well with a decrease of 28 units, however, the second only showed a decrease of 7. However, this is not surprising since the dataset is very small. Regarding the interface, the participant prefered not to see the real-time data for the reason that I suspected, it was too distracting.

Iteration 3

 

Although there is evidence that specific genres and types of music will have a universal relaxing impact on one’s body, a number of studies have also shown that when exposed to one’s preferred genre of music, the indicators of relaxation can significantly increase and anxiety may decrease compared to when exposed to other unfamiliar music (Bernardi, 2005; Salamon, 2003). Therefore, for the third iteration, I decided to let the user choose their own relaxing song that they are familiar with. In addition, I changed the interface based on the user feedback and enabled an option to hide real-time data. Also, to improve the user experience and the effectiveness in stress reduction, I added a breathing guide that tells the user when to inhale and exhale. Such diaphragmatic breathing can significantly reduce stress after just a single practice (Arsenio & Loria, 2014). Although I was not able to implement it due to time and skills limitations, I imagine it to send light vibrations via the phone or a wearable device indicating when to inhale or exhale. However, I asked the participant to take deep breaths during the listening session.

 

The participant chose a song that they personally find relaxing and listened to it while taking deep breaths. The song performed rather well, GSR went from 280 to 230 in the first 40 seconds, likely due to the deep breathing too.

Listening to a personally preferred relaxing song while taking deep breaths.

 

REFLECTION AND CONCLUSION

Overall, the process of testing GSR in response to music turned out to be trickier than I thought. It is difficult to ensure the participant is in a stressed state over and over again as they become habituated to the stressor. There are so many other factors that should be accounted for to ensure accuracy in testing, such as room temperature, testing time etc. After doing the experiments, GSR seems to be quite an appropriate indicator of one’s emotional arousal in response to music, however, it would be more powerful if it was combined with other biosensors such as HR monitor. The recommendation system should be tested further with a larger and better organised data set but I think having quite an extensive variety of song characteristics to train it with was very helpful. Lastly, incorporating breathing techniques proved really effective in reducing stress. There is potential to make it even more engaging and data-driven by combining the breathing guide with HR data.

REFERENCES

APA. (2018, October). STRESS IN AMERICA<sup>TM GENERATION Z</i>. Retrieved from https://www.apa.org/news/press/releases/stress/2018/stress-gen-z.pdf

Arsenio, W. F., & Loria, S. (2014). Coping with Negative Emotions: Connections with Adolescents’ Academic Performance and Stress. The Journal of Genetic Psychology, 175(1), 76–90. https://doi.org/10.1080/00221325.2013.806293

Bernardi, L. (2005). Cardiovascular, cerebrovascular, and respiratory changes induced by different types of music in musicians and non-musicians: the importance of silence. Heart, 92(4), 445–452. https://doi.org/10.1136/hrt.2005.064600

Coulon, S. M., Monroe, C. M., & West, D. S. (2016). A Systematic, Multi-domain Review of Mobile Smartphone Apps for EvidenceBased Stress Managementsa. American Journal of Preventive Medicine. https://doi.org/10.1016/j.amepre.2016.01.026

Dallman, M. F., Strack, A. M., Akana, S. F., Bradbury, M. J., Hanson, E. S., Scribner, K. A., & Smith, M. (1993). Feast and Famine: Critical Role of Glucocorticoids with Insulin in Daily Energy Flow. Frontiers in Neuroendocrinology, 14(4), 303–347. https://doi.org/10.1006/frne.1993.1010

Graupensperger, S., Cadigan, J. M., Einberger, C., & Lee, C. M. (2021). Multifaceted COVID-19-Related Stressors and Associations with Indices of Mental Health, Well-being, and Substance Use Among Young Adults. International Journal of Mental Health and Addiction. https://doi.org/10.1007/s11469-021-00604-0

Hazen, E. P., Goldstein, M. C., Goldstein, M. C., & Jellinek, M. S. (2011). Mental Health Disorders in Adolescents. Amsterdam, Netherlands: Amsterdam University Press.

Kessler, R. C. (2012). The Costs of Depression. Psychiatric Clinics of North America, 35(1), 1–14. https://doi.org/10.1016/j.psc.2011.11.005

Mahani, S., & Panchal, P. (2019). Evaluation of Knowledge, Attitude and Practice Regarding Stress Management among Undergraduate Medical Students at Tertiary Care Teaching Hospital. Journal of Clinical and Diagnostic Research. https://doi.org/10.7860/JCDR/2019/41517.13099

NCPSB. (2022). AD: Tachtig procent van jongeren zit door corona tegen burn-out aan. Retrieved from https://nationaalcentrumpreventiestressenburn-out.nl/ad-tachtig-procent-van-jongeren-zit-door-corona-tegen-burn-out-aan/

OECD. (2017). Most teenagers happy with their lives but schoolwork anxiety and bullying an issue. Retrieved from https://www.oecd.org/newsroom/most-teenagers-happy-with-their-lives-but-schoolwork-anxiety-and-bullying-an-issue.htm

Student Development, 51(1), 79–92. https://doi.org/10.1353/csd.0.0108

Stults-Kolehmainen, M. A., & Sinha, R. (2013). The Effects of Stress on Physical Activity and Exercise. Sports Medicine, 44(1), 81–121. https://doi.org/10.1007/s40279-013-0090-5

Read more "Combining GSR data and music to reduce stress"

Identifying Invasive Plant Species Using Deep Learning

Premise How can a deep learning image classification tool be used to identify invasive plant species? Synopsis Biological invasions, where a non-native species is introduced to a new environment where it proliferates and dominates, i.e., becomes invasive, drastically altering the functions of that ecosystem, are one of the biggest drivers of biodiversity loss globally. Once […]

Read more "Identifying Invasive Plant Species Using Deep Learning"

Awareness for the invisible: Predict healthier journeys from air pollution data

Premise Creating healthier cycling commutes using crowdsourced IoT GitHub: https://github.com/philffm/FreshAir Figma Concept: https://www.figma.com/proto/ Synopsis The global climate and health crisis is affecting everyone and from 2030 and 2050 is expected to account for “250 000 additional deaths per year.” (World Health Organization (2021) Air pollution, which poses a major challenge in most emerging economies, is […]

Read more "Awareness for the invisible: Predict healthier journeys from air pollution data"