Guài  (2023)

Project Summary

Guài means strange, or monster, in Mandarin. The project started with the question – if AI could really decipher our personality, what sort of inner monster would we find inside?

Commissioned for Melbourne Fringe 2023, the work was presented at Footscray Community Arts from 5 - 15 October 2023. Although the work was targeted toward youth of ages 10+, the audience came from a large demographic spread across diverse ages and cultural background, particularly due to the location of Footscray Community Arts within a large immigrant and multicultural community. This was particularly pertinent in the context of well-documented biases against cultural minorities in the field of biometric analysis (Michael et al. 2022).

As participants individually entered our "portal", they were confronted with an AR mirror (a translucent two-way mirror sitting atop an LED screen) allowing them to see a reflection of their own body overlaid over a virtually-constructed reality. An Azure Kinect computer vision camera was positioned above the mirror, allowing us to track the participant body and adjust the virtual view with their head position. The camera was also used to capture a photo of the participant for biometric facial analysis.

Facial recognition AI provided by Amazon Rekognition ("Guidelines on face attributes" 2023) was used to obtain general information such as age and gender. 

During the development stage of this work, we used Microsoft’s Azure Face for emotion recognition. However, Microsoft discontinued public access to the API in June 2023 due to concerns about the ‘lack of scientific consensus on the definition of emotions, the challenges in how inferences generalise over use cases, regions and demographics and the heightened privacy concerns around this type of capability’ (Crampton 2022).

​

Amazon Rekognition also provided the ability to extrapolate 8 different emotional states being happy, sad, angry, surprised, disgusted, calm, confused and fear as well as a confidence rating for each metric. 

Amazon Rekognition provides a binary prediction of gender (male/female). In its documentation, it states that the prediction is based on the physical appearance of a face in a particular image and does not indicate gender identity. In Guài, we chose to retain rather than remove this prediction despite its non-inclusive nature to make transparent the types of biometric analysis and categorisation that are we are potentially subject to without our knowledge or consent. Similarly, predictions of emotional expression are based on the physical appearance of a person’s face. In Amazon Rekognition’s documentation, it states that this does not indicate a person’s actual internal emotional state. For example, a person pretending to have a happy face in a picture might look happy but might not be experiencing happiness.

For Guài, we took creative license to extrapolate personality traits from these emotional states using an algorithm provided by ChatGPT - being happiness, boringness, intelligence, attractiveness, emotional stability, weirdness, trustworthiness and egoism. In reality, the algorithm provided by ChatGPT had no scientific basis. We also had no way to interrogate how ChatGPT derived this algorithm, due to the black box nature of complex neural networks (Bathaee 2018). However, we used this AI construction to challenge participants to think critically about the way in which society tends to accept AI as omniscient and accurate despite their many documented biases, particularly against bodies which are transgressive, coloured, or non-normative (Singh et al. 2021).

Based on the detected personality traits, we assigned participants to one of eight virtual avatars derived from monsters described in the Chinese classic Shan Hai Jing (Classic of Mountain and Seas), a compilation of mythic geography and myth (Chen Cheng jin et al. 2010). For example, participants with a very high "attractiveness" score would be assigned the Tian Hu, described in the literature as a shape-shifting tempter. Participants with a very high "egoism" score would be assigned the Tao Wu, described as a vicious and stubborn beast. The participants were allowed 3 minutes to "embody" the monster, as the tracking camera overlaid the virtual avatar over the participants’ reflection in the AR mirror. In addition, a soundscape was created for each individual participant based on the personality traits. The soundscape was interactive, encouraging participants to become the monster through a combination of kinaesthetic, visual and sonic senses.