2022-ongoing / digital, interactive, documentary poetics
Image description: Work in progress of The Cyborg’s Prosody. The images shown depict excerpts of Level 1 of the game, which includes visual artwork against white and yellow backgrounds with black text. Illustrations by artist Christina Dacanay.
The Cyborg’s Prosody is an interactive docu-poetics artwork designed and optimized for mobile use. Similar to the way a user must validate their voice for various smart technologies, the platform will require the same of the player. For example, this validation entails the player reading excerpts of French film theorist and composer Michel Chion’s work (in English) focused on the disembodied voice or from Cathy Park Hong’s book Minor Feelings. Once a player has completed “voice authentication,” they will be prompted to select the first (and easiest level) of stories. As the stories and levels progress, the player is expected to mimic Tagalog words and phrases to advance in the game.
The stories have varying levels of difficulty. The easiest level (first) entails repeating excerpts of a story they are listening to in English with a reader who has an discernable Filipino accent. The most difficult level (fifth) requires the player to repeat excerpts of a story told completely in Tagalog. In order to advance in the game, the player must repeat the story in the language and tonal quality of the reader. The player is being inducted into speaking the Cyborg’s prosody, which is inspired and modeled by a Filipino speaker.
Each story, or level, is written based on a transcript of an interview I conducted with my mother on her experiences of immigrating to the United States from the Philippines. The entire work consists of five stories based on the stages of grief and loss: 1) denial, 2) anger, 3) bargaining, 4) depression, and 5) acceptance. In particular, the grief that accompanies the loss of a mother tongue. The computational part of this project involves the development of machine learning classifiers to create a speech corpus based on my mother’s voice, affect, and accent.
Images of Creative Process and Ideation
Image description: Phase II mock ups of The Cyborg’s Prosody. The left hand image shows the listening interaction against a white background with black and gray text. The right hand image shows the speaking interaction of the game, which includes a yellow background with a combination of black and white text. Illustrations by artist Christina Dacanay.
Image description: Phase I mock ups of The Cyborg’s Prosody during the research and development phase. The image to the left shows a set of three iPhone screens with black text and buttons. The image on the right hand side shows three iPhone screens with black and red text.
Credits (and Immense Gratitude):
Advisors and Mentors: Lillian-Yvonne Bertram, Johann Diedrick, and Robert Ochshorn
Collaborators: Christina Dacanay (Illustration and UX Design)
Abram Stern, Ph.D. (Web & Software Development)
The Cyborg’s Prosody is made possible through the generous support of the Mozilla Foundation Creative Media Award, The Frank-Ratchye Studio for Creative Inquiry, Duolingo, and the Banff Centre for Arts and Creativity.
Dorothy R. Santos, Ph.D. 