The Sentinel Self is an interactive artwork that reflects on the entanglements between human immune systems and environments increasingly polluted by microplastics. Built within a video game engine, The Sentinel Self presents the audience with an immersive world containing an ongoing and complex simulation of hundreds of interacting cell-like creatures within a vast aquatic, body-like ecosystem.

For this work I collaborated with Sissel Marie Tonn, developing a generative sonic landscape for the simulation based on a collage of real-time deep-neural-network voice conversion models. The different agents within the simulation are given voice by using human vocal performances transformed through these conversion models. The conversion models use as their source material a cacophony of vocal sounds across species – from sea gulls to marine mammals to Chinese opera.

Additionally, I worked together with Sissel and her scientific collaborators in Trieste as an interdisciplinary intermediary between artistic and scientific discussions. Helping to develop the simulations of the work, and especially the parts of the simulation that rely on synchronicity phenomena such as those observed in organisms and organismic systems – fireflies, brain networks, and menstrual cycles.

More information on the work at Sissel’s website.

Credits

Created by Sissel Marie Tonn
Lead Unity Developer – George Simms
3D world building and design – Zuza Banasińska
Music and Sound – Jonathan Chaim Reus
Interaction designer and hardware developer – Sam Bilbow
Rigging and animation – Sarah Fernandez
3D Modeling – Sissel Marie Tonn
Project support – CheeYee Tang
Texts – Heather A. Leslie & Sissel Marie Tonn
Webdesign Wikimedia- DXR Zone
Immunology advisor – Juan J. Garcia Vallejo (VU Amsterdam)
Scientific advisors Area Science Park – Alessio Ansuini, Francesca Cuturello, Alberto Cazzaniga, Lisa Vaccari

With the support of: S+T+ARTS, Area Science Park, Stroom Den Haag, Creative Industries Fund NL, The Sussex Centre for Consciousness Science