Ann Li

is a multidisciplinary experience designer and researcher exploring experiential narratives and connective systems

Best viewed on desktop

Ann Li

Neurobiologist-turned designer driven by sense-making, systems thinking, and storytelling. Seeking full-time interaction, experience, or product design roles.

Work spans the digital and physical, including immersive experiences, functional user interfaces, and creative direction and strategy.

If walls could speak

A sound-based installation evoking curiosity and giving form to ambient presences

Role

Concept Ideation
Interaction Design
Narrative Development
Physical Fabrication
Machine Learning Development

Context

Spring 2023

Tools

Arduino
RFID
Fusion360
3D Printing
Audition

What traces of our digital presence are unintentionally left behind, and how might they take on a life of their own?

If walls could speak gives form to contradictions between the transient, yet indelible, nature of our digital footprint.

As the divide between cyberspace and reality blurs, the disembodiment of data raises questions around what it means for the origin–and transmission–of information to be untraceable. The spillage of digital technologies beyond the screen is inevitable, whether we perceive it or not. Despite this, we expect networked technologies to behave in certain assumed ways– ways that, when subverted, create a deep sense of unease.

Haunted Smart Home

An interactive installation taking place in the walls of a tech-enabled haunted house.

When our private thoughts and conversations inadvertently leak; when our every move is observed, documented, and predicted; when we gain access to the most intimate aspects of others’ lives; when representations and recordings of our past selves haunt us from beyond their time– using the power of sound to mediate connection over time and space, If walls could speak explores these ambient presences and aims to revive ghostly digital pasts imprinted within these walls.

Prototype

If walls could speak is an installation of two parts– a panel of walls with hidden RFID tags embedded within it, and an enchanted (tech-enabled) cup.

This cup is equipped with an RFID sensor, microcontroller, DF Player Mini, amplifier, potentiometer, speaker, and battery. Each RFID tag ID links to a unique mp3 sample, with 13 tags adding up to 5 minutes of audio total.

The contents of each audio sample range from lighthearted cafe conversations, to deeply personal heart-to-hearts and cathartic tell-alls, intended to reflect the variable, personal, unpredictable nature of leaked digital presences.

These audio recordings were sourced from across the world, either from personal archived memos or from consenting friends and acquaintances. Because of the sensitive nature of this content, all voices were altered in Adobe Audition, a distortion that was intensified through the lofi effect of the bone conductor transducers used.

Intention

Designing an installation that taps into a common form and interaction modality, while serving a different function and operating under a series of unfamiliar operations.

For this exploration, I was curious as to how I might blend an object-oriented design approach while pulling from cultural mythologies and otherworldly references. This was originally inspired by my personal fondness for voicemail and perceived lack of remote and ambient presence to mediate communication. Voicemail is “obsolete”, yes. But during a current analog revival and revolt against hyperreality and instant gratification, could it make a comeback? If our relationships and expectations of this mode of communication have changed, what could they be instead and what would that afford? I’m inspired by my personal interest and experiences with the voice memo/answering machine message, but this is augmented by others’ experiences as well- there are countless examples of people saving voice messages from loved ones when apart, or of those who have passed away. Often memories of voice are the first thing to go, the first thing people miss and wish they had access to. These observations all inspire this concept.

Context

Experimenting with new technologies to produce outputs that represent past interactions.

I found myself exploring concepts around telepresence as ghostly or otherworldly presences “trapped” or “residing” within objects, ranging from enchanted (otherworldly, awe-inspiring, +) to possessed (unsettling, haunting, -). I drew initial inspiration from Durrell Bishop’s Marble Answering Machine concept, which explores the embodiment of voice messages and offers an interesting tangible alternative to familiar interaction sequences.

I wanted to experiment with RFID, which lent itself to the localization of disembodied sounds as outputs. Because I imagined this exhibit to be situated in an entryway or somewhat open setting, the design needed to account for perceived risk-taking or vulnerability during use. I’ve been inspired by classic horror tropes involving the idea of mysterious “voices in the walls” or haunted recordings/possessed objects. Since this was going to be situated in the Haunted Smart Home, I wondered what it might be like to imbue the house itself with some living presence. Initially, this took form in a Monster House-style concept, whereby the house itself is alive and sentient, with a specific agenda that is projected onto its inhabitants. In exploring the affordances of disembodied voices and sound as a hallmark of past events, work such as Daniella Petrelli’s FM Radio: Family Interplay with Sonic Mementos served as great reference. The final concept blends these precedents with common cultural references, such as using an overturned glass to eavesdrop and hear through walls.

Process

The initial concept evolved quite drastically throughout the course of development, retaining key aspects around technical implementation and the core theme of ambient, disembodied audio presence. From a preliminary answering machine form came the “listening glass” motif, which I swapped out with ceramic mugs to ensure all internal components would be hidden from view. After assembling the main circuit (Arduino, RFID sensor, DF Player Mini, speaker), I checked for functionality with external power sources (LiPo batteries), added various peripherals to enhance the flexibility of the system across different contexts (potentiometer to adjust volume, audio amplifier), and finally swapped out the speakers for bone conductor transducers to allow for a more intimate, visceral final interaction.

‍

A series of four housing structures and fixing mechanisms were prototyped and tested, with the final version using a flexible semi-solid housing 3D printed with TPU and covered with a semi-translucent acrylic, held in place with generic office putty. Though I wanted this experience to be quite intimate and personal, to prevent it from being a single-person installation, I wanted to create 2 cups for alternating use. Since I only had 1 Arduino Nano 33 BLE Sense, I sourced an Arduino Nano for the second cup, which required different code and wiring to initiate serial communications. After sorting this out, things were working well until the Nano BLE Sense became unresponsive. After attempted troubleshooting and conferral, it was determined that the board was unusable. I rewired the first cup and scavenged another Nano to swap in, but suddenly faced issues with the boards not retaining uploaded sketches upon connection to LiPo batteries. Upon experimentation, the cups became functional when connected to conventional 9V batteries, but lacked secure circuit connections due to time constraints. The final version of the installation, though functioning and responsive, was quite delicate as a result and required frequent refreshes and debugging.

‍

Open Questions and Next Steps

Questions and considerations that this exploration aims to explore include:

The disembodiment of data and what it means for the origins of information to be unknown
The ways we think about and expect technology to mediate social connection, memory, information sharing, and communication– through an artifact that revives and reimagines what might have otherwise been lost or unknown.

How do we make our presence known in our digital footprint? What “ghosts” of ourselves are we leaving behind unintentionally? How remote and ambient presences can mediate communication and connection, and how can we tap into the power of sound to surface memories and reminiscence? What it means for a non-human entity to embody and take on animistic qualities (through sound and storytelling)?

To further explore these questions, a more distinct connection between the content of the outputs and the manner of soliciting outputs could be considered. For many visitors, the exact nature of the voices they were hearing was unclear–but a point of great curiosity and interest. When I explained the exact origins of each message–and the deeper context–the experience became not only more entertaining and voyeuristic, but more personal as well. Rather than only peering into the lives of others, visitors were able to make connections to aspects of their own lived experience as well, twisting the original intention of the concept in an interesting direction.

Reflection

Overall, the embodiment elicited reflected the experience desired, with folks crouching and standing up on their toes to explore the full scale of the walls. Noise levels were well accounted for by the amplifier and volume dials, and when combined with the object labels and exhibit description, the overall narrative and backstory was effectively communicated to many visitors that I spoke with afterwards. Ideally the nuances of use (place the cup firmly and flush against the wall, slide and rotate it around to find the tags), rather than needing to be explicitly called out, could be addressed by refining the system itself to allow for greater flexibility of user interpretation and use.

Due to lack of stability and last minute hiccups, public reception to the demo was limited. Those that were able to encounter the functioning installation, however, delighted in discovering the different hidden audio samples and found the concept alluring and intriguing. For future iterations, either the sensitivity of the RFID sensor should be heightened, or the location of tags made more obvious. Soldering all connections to a flexible proto board would also potentially make for a more resilient artifact. Though I wanted participants to have to search around for the hidden audio presences, it was a bit too difficult for visitors to activate the audio samples during the live demo, which would have resulted in a higher drop off rate as people became overly frustrated or bored. Potential refinements could include improvements as simple as a patterned wallpaper overlay to mark active areas, or further interactivity such as tag locations which respond with light or change in form.

Link to full case-study and documentation

Inspiration and References

Daniella Petrilli, FM Radio: Family Interplay with Sonic Memories

Durrell Bishop, Marble Answering Machine

Echelle Inconnue, Listening to the walls whisper

A Changing Arctic, Carnegie Museum of Natural History

XR Augmented Learning Tools

Ann Li 2020

Don't be a stranger

If you think I'd be a good fit for your project or team, let's connect.