Premise

Creating an AR tool to highlight personal recommendations from friends in the algorithmic recommender system Netflix.

Synopsis

Personal recommendations from friends often get lost in between messenger services and are rarely being made use of. To make them accessible for the user at the point of need, AR can be used to facilitate that interaction in an immersive and novel experience for the user. Instead of choosing from algorithmic recommendations selected by Netflix, users can bond over recommended content with the help of an AR feature, and therefore form a social connection that is not inherent in the way we use recommender systems right now.

Substantiation

The social connection created between two people who were sharing personal recommendations with each other has been taken over by algorithms. As users are increasingly relying on Big Tech’s black boxes to recommend them the next movie to watch, a podcast to listen to, or which restaurant to visit, personal relationships often get neglected. Research in User Experience Ecosystems found out, that 80% of people recommend items like movies, songs, events, etc. to their friends on at least a weekly basis. However, only about 20% of people note down recommendations they received in their notes to use them at a later time. The majority of people reported that it takes them too much time and effort to find recommendations that got lost in between chat messages on various different platforms and messenger services.

Roco – a student-led research group and start-up – tries to solve this issue by providing a platform for sharing and storing personal recommendations, keeping track of their consumption, and giving and receiving feedback from friends.

This research solely focuses on one of the interactions in the whole user experience of the product, which is providing users with social recommendations from Roco in the context when they actually need them. For instance, a user looking for a movie to watch on Netflix is likely to experience choice overload due to a large number of options to choose from (Gomez-Uribe & Hunt, 2015). The specific objective of this research is to explore how an augmented reality (AR) feature may help to choose by integrating a social component.

While 73% of internet users use their smartphone as a second screen while watching TV already (Global Web Index, 2022), AR was chosen as a technology to facilitate social interaction between people (Shu et al., 2018). Adam & Pecorelli (2018) showed, that recommendations from other users seen in AR positively influence the selection of items presented. As AR transforms the users’ experiences with regular products and services into creative playgrounds (Jessen et al., 2020), it can allow users to experience their friends’ recommendations in a novel way to find the content they enjoy.

To illustrate the use of AR in Roco, and test, if virtually bringing friend’s recommendations into the same room can help users make a decision faster and with more satisfaction, three prototypes were created to evaluate the role, implementation, and look and feel.

Prototypes and experiments

1^st iteration

The 1^st iteration is testing the role of the AR feature by investigating what it can do for users, and how it can benefit them in its usage. A storyboard was created to find out if users want to use an AR feature that shows the recommendations of their friends by scanning the Netflix logo on their second screen (usually the TV or PC). It was shown to 5 Roco users to get their feedback on the proposed solution.

The key findings of this iteration are:

• Users already write down recommendations from their friends, but then have to go through an extensive list to see which of those movies are available on Netflix
• Users already use smartphones next to using Netflix on TV to look up IMDB ratings
• Users are missing that feature in Netflix and would like to have it integrated

2^nd iteration

The 2^nd iteration is testing the implementation of an AR feature on the Netflix dashboard with Artivive, a platform for augmented reality solutions. After selecting a vector graph of the Netflix logo, it was imported into Artivive to work as a target image for the AR feature.

The Netflix logo itself is already in the database and is being used as a target image for a different AR feature. Cutting off the logo in smaller pieces did not work either, as the image would not be recognized due to it being too minimalistic. Some pieces of the logo would score higher recognition speed scores than others, but none would work entirely.

After running into more problems with technical issues on Artivive itself, further iterations of the prototype took place on UniteAR, an AR creation platform with more functionalities than Artivive. While the Netflix logo – in any shape or form – would not get processed on UniteAR either, the Netflix profile menu would work.

 

Implementing the AR feature at this point in the user journey has the benefit of giving the user the chance to find a quality recommendation from a friend before diving into the vast amount of algorithmic recommendations on the Netflix dashboard. The user can therefore already have an endpoint to reach when opening the app and can direct to the preferred content right away.

3^rd iteration

The 3^rd iteration investigates the look and feel of the AR feature to simulate interaction with it. Designing the tiles for the personal recommendations took place in Figma. As the tiles were barely visible in the 2^nditeration, they got a higher opacity to stick out from the background.

Next up was the placement of the individual recommendation tiles in the AR space. To find out, how the tiles could be placed, two different versions of the AR feature were developed and tested with 5 Roco users. The first version showed the recommendation tiles in the screen, overlaying the profile menu. The second version showed the recommendations around the profile menu.

 

All of the participants of the test preferred the second version over the first one, as the recommendations were bigger, therefore more readable. Furthermore, participants stated, that the size and the position of the tiles in the room instead of on the screen made the experience more immersive.

As a final step, the personal recommendation tiles have been made interactive. By clicking on a recommendation from a friend, the user is redirected to Roco, where they can see the personal note for that recommendation. After reading through it, they can open up a link for the item in Netflix itself to either read the description or stream it from their phone to their TV.

 

When tested with 5 Roco users, they all found the AR feature to be a helpful tool to get access to their friends’ recommendations and stated, that using it could potentially help them find enjoyable content faster. Users reported to see it as a novel function, that adds to the immersion, but they would not necessarily use it as their go-to feature. One user stated that they would use the AR feature to scan content to get the opinion of their friends.

Conclusion

This research set out to explore how an AR feature may help to choose content on Netflix by integrating a social component in form of personal recommendations from Roco. The results of 3 iterations on role, implementation, and look and feel show that users are inclined to use the proposed AR feature to find new and interesting content on the platform through their friends. These findings illustrate how AR can be utilized to give users a novel way to experience their friends’ recommendations. A further study could assess the implementation of a content-specific AR feature, that shows users their friends’ opinion on a previously recommended item, as suggested by a user in the last iteration.

References

Adam, M., & Pecorelli, M. (2018). Recommendations in augmented reality applications – The effect of customer reviews and seller recommendations on purchase intention and product selection. https://aisel.aisnet.org/ecis2018_rp/6

Global Web Index. (2022). Second Screening Infographic.

Gomez-Uribe, C. A., & Hunt, N. (2015). The Netflix Recommender System: Algorithms, Business Value, and Innovation. https://doi.org/10.1145/2843948

Jessen, A., Hilken, T., Chylinski, M., Mahr, D., Heller, J., Keeling, D. I., & de Ruyter, K. (2020). The playground effect: How augmented reality drives creative customer engagement. Journal of Business Research, 116, 85–98. https://doi.org/10.1016/j.jbusres.2020.05.002

Shu, J., Kosta, S., Zheng, R., & Hui, P. (2018). Talk2Me: A Framework for Device–to–DeviceAugmented Reality Social Network. IEEE International Conference on Pervasive Computing and Communications (PerCom).