Leonardo

The Case for Intercorporeal VR

VR is typically seen as a product for private consumption. Consequently, many public exhibitions neglect the site-specific requirements of the exhibition space. Often, a VR headset is placed in a gallery with limited consideration of the social context and no connection to the outside world. Such experiences rarely bring those outside the headset into the experience. Body RemiXer is designed to activate bodies as a site for social exchange, acting as what Nicolas Bourriaud calls a “social interstice” [1]. This focuses immersants toward their intercorporeal relationship with one another and mediates the social norms of the space. Intercorporeality is the relational connection between bodies, as described by embodied mind philosopher Maurice Merleau-Ponty. Merleau-Ponty argues that our social interactions are fundamentally embodied and that we understand others through the possibility of mirroring their actions through our own bodies [2].

Body RemiXer is a response to the emerging social future that promises to make reality ever more virtual. The installation attempts to counter telepresent mediation that promises to facilitate social connections while simultaneously dividing us into isolated physical spaces. In 2002, Bourriaud observed that “communications are plunging human contacts into monitored areas that divide the social bond up into (quite) different products . . . before long, it will not be possible to maintain relationships between people outside these trading areas” [3]. This claim has become especially true with the emerging popularity of VR.

Instead, Body RemiXer is a social experiment that explores how VR could help connect people through their copresent bodies. We wanted to demonstrate the importance of this embodied aspect of the social experience and the capacity of VR to activate bodies and alter social norms to facilitate connection.

Inspiration

There are some examples of VR being used for colocated social interaction. CAVE brings all immersants into the same co-located virtual space using avatars [4]. Despite being a passive experience, the presence of audience members acknowledges the social context and allows participants to share in the experience. Invisible Walls investigates copresence between an immersant wearing a headset and spectators represented by abstract forms [5]. This interaction enables the immersant to feel copresent with the spectators and encourages interaction between them, allowing for connection at a distance. Body RemiXer takes this colocated interaction further, bringing the headset-wearer into the same space as the spectators, enabling them to interact on the same level. Their bodies become activated as the central element of the installation, drawing immersants’ attention toward one another.

Body RemiXer was inspired by The Machine to Be Another [6], which allows immersants to see another’s perspective through synchronized movement. While this experience can give immersants the sense of being in another body, it simultaneously draws attention to the similarities and differences between those bodies, dividing them through the same process it uses to connect them. Body RemiXer was designed to retain the focus on the physical body without reinforcing superficial appearances. We also wanted to use the abstract appearance to allow immersants to freely move from the boundary of their body to connecting with another, and back again. [End Page 395]

In We Are All Made of Light [7], Maja Petric captures bodies as simple “shadows” made of light that form evolving constellations to create a sense of unity and interconnectedness through abstraction. We were inspired by this concept and the aesthetic of Notes on Blindness [8], which demonstrated the capacity of abstract representations to represent the essence of human presence in VR. Similarly, we created an abstract neutral body that revealed only a silhouette or “aura” of the immersants and removed most visually identifying features. We hoped that this abstraction and neutral figure would focus immersants on the body and movement, encouraging new forms of connection that go beyond the surface.

The Experience

A critical aspect of Body RemiXer is creating an experience that suits a variety of levels of curiosity and openness to public expression. We want to encourage individual exploration that progresses toward tactile connections with other immersants. Immersants can interact with the system via either mirror-like projections or the VR headset. Immersants can play with their particles’ movement individually; however, to fully realize the potential of the system, they must interact with other immersants. Immersants can selectively move between three modes (Fig. 3); the particles respond differently in each mode, guiding immersants’ attention to their engagement, connection and synchronization with others. In solo mode, immersants’ individual aura bodies are distinct and well defined, giving them an opportunity to embody the aura. In exchange mode—which immersants enter by exchanging a high five with a partner of their choosing—the particles flow between immersants, representing their connection. Finally, in swap mode—which immersants enter by touching both hands to their partner’s hands—each immersant’s particle body is overlaid onto the other’s, focusing immersants’ attention on the relationality between their body and their partner’s.

The experience begins from the moment onlookers notice the installation from afar. They can watch others play and interact with the piece, becoming spectators of an impromptu performance, and can enter whenever they desire.

________

Discovering the aura in the mirror

Upon entering the active space, immersants see their ethereal reflection fade into existence in the projection as their bodies are detected by the Kinect. Particles swarm around their body, appearing as

a kind of virtual aura. Each immersant’s aura takes on a different color, allowing them to retain a sense of identity that can later be mediated in the other modes. The immersant can play with their reflected aura, responding to the particles’ motion and experimenting with different movements. This allows the immersant to transition into the active space of the installation, establishing a connection with their own virtual body. The aura was designed to allow immersants to embody their virtual bodies by removing biasing features, such as sex or race, that might alienate other immersants, and to encourage playful interaction through visuals that respond to their movement energy.

Becoming more immersed

We designed Body RemiXer to allow immersants to interact with or without the headset to make the installation more accessible to all participants. Those who put on the headset find themselves surrounded by the virtual space seen in the mirror projections. Their body and the bodies of those around them are transformed into auras of energetic particles, each with their own color. The identities of friends, strangers and oneself become blurred together as they are abstracted.

Click for larger view
View full resolution

Fig. 3.

Progression of interaction through Body RemiXer.

(© John Desnoyers-Stewart, 2020)

[End Page 396]

Connecting through touch. As an immersant seeks to understand this new reality, they reach out toward these strange bodies. Another immersant reaches toward them in response and, as their hands make contact (Fig. 4), they find their aura bodies suddenly connected as they progress to exchange mode. Particles fly between their bodies, breaking the previously clear boundary and creating a sense of connection.

Moving together

Once connected, the immersants hear a faint drumbeat. The drumbeats are heard equally by all and shared between pairs. Each pair has a unique track, so they contribute to the overall drums in layers. Through their shared movement, they discover that they can control the loudness of the drums. Moving synchronously makes the drums grow louder, while moving asynchronously from one another makes the drums grow softer. This audio feedback is intended to encourage immersants to move in response to one another, further strengthening their connection.

Swapping bodies. By touching both hands together, immersants can move from exchanging particles between their bodies to an abstract form of body swapping. Each sees their partner’s aura superimposed onto their own. This allows immersants to see their body and their partner’s simultaneously, reinforcing the similarities and discrepancies between them and encouraging attention to their partner and movement synchronization.

A Carnival of Mixed Realities

In its first public exhibition, Body RemiXer was experienced by 400 attendees at A Carnival of Mixed Realities: Fun Palace Edition (Fig. 5). This one-night mixed reality event sought to reimagine Littlewood, Price and Pask’s concept of the Fun Palace, a reconfigurable and active space of inquiry designed to stimulate social change by prompting new possibilities [9]. The event was designed to facilitate unconventional social interventions like Body RemiXer. Attendees were primed to expect the unexpected and to engage with the installations, as everything in the space was interactive. We observed attendees approach Body RemiXer with active curiosity, exploring the possibilities offered by the installation. Attendees could freely enter and leave the space, without waiting in line, and could interact through the projections until the headset became available.

Click for larger view
View full resolution

Fig. 4.

Physically touching their hands together allows participants to connect their virtual auras and move between the installation’s three modes.

(© John Desnoyers-Stewart, 2020. Photo by Stephen Crocker.)

Click for larger view
View full resolution

Fig. 5.

Body RemiXer as shown at A Carnival of Mixed Realities.

(Photo © Andreas Psaltis, 2019)

Activating Bodies as a Site for Social Exchange

Body RemiXer is what Bourriaud refers to as a relational artwork: a work that takes human relations as its subject and forms a “social interstice”—a state of encounter that runs counter to the norms of everyday life. It mediates immersants’ bodies virtually with material effects on their social relations. It is “an art form where the substrate is formed by intersubjectivity . . .” [10]. Somaesthetic designer and interaction researcher Kristina Höök claims that this intersubjectivity is driven through an intercorporeal process that can be mediated through technology [11]. The subjective and affective experience of social interaction is thus experienced through the relationality between bodies as attuned by the medium of interaction. [End Page 397]

Body RemiXer capitalizes on this intercorporeality to gain an alignment of immersants’ inner states by aligning their physical states and giving rise to an intersubjective experience. As Höök claims, “In social relations, the other’s body influences us in an immediate sense; we mirror and might feel what they feel” [12]. In Body RemiXer, immersants observe and respond to one another’s movements, establishing the body as a site of social exchange, through an intercorporeal relationship that extends into future connections.

As installation artist Nathaniel Stern claims, interactive artwork reframes the body’s movement, thoughts and sensations: “Interaction is a limitation—but it is also an amplification. At its limits, interactive art disrupts our relational embodiment and thus attunes us to its potentials” [13]. In Body RemiXer, certain social elements are hidden or subdued, amplifying others. The body’s appearance is altered and the others’ gaze is hidden, removing visual indicators typically used to discriminate and replacing them with a unifying aesthetic intended to transcend physical boundaries. Through its imagery, Body RemiXer restricts the forms of social interaction available to immersants to attune them to aspects that ordinarily go unnoticed and momentarily freeing them of their conventional social boundaries.

This momentary alteration has the potential to persist beyond the installation, both in those relations established through the experience and in the newly realized social potentials of the immersant. As an MR installation, Body RemiXer is designed to allow for connection to evolve across the virtual/actual divide, connecting the experiences within these realms together. Immersants are encouraged to adopt their virtual auras through MR mirrors (Fig. 6). From here, interpersonal connection starts as physical touch between immersants and extends into the virtual merger of their auras. This could then extend back to the connection between the actual participants leaving the exhibit and beyond into their subsequent relations.

Observations on How Virtual Reality Can Mediate Social Interaction

Body RemiXer is a prototypical social structure that forms a template for what VR-supported intercorporeality could look like. By inverting many of the assumptions about what VR is, we explore what it could be. To understand the social effects of Body RemiXer and the ways it mediates intercorporeal relations, we discuss our observations of immersants’ interactions with the system and with one another below. We observed immersants’ behavior directly while facilitating the installation and later through video footage of the Fun Palace event. Exemplary images from our video footage can be seen in Fig. 7.

Divergent Social Norms

The altered relations between immersants showed that the context of the installation disrupted social conventions, establishing new ones in their place. Immersants’ performances may have been normalized by the setting and established a space of heightened intersubjectivity that drew their attention toward one another. From the perspective of the headset-wearer, those norms were further altered, creating a space with two disparate social norms overlaid onto one another. Headset-wearers were seen reaching out to touch the mirrors and the virtual bodies surrounding them, perhaps to reconnect with the physicality of the space and to test whether it was “real.”

Click for larger view
View full resolution

Fig. 6.

Immersant reaching toward their reflection in VR.

(Photo © Melissa Dex Guzman, 2019)

[End Page 398]

Click for larger view
View full resolution

Fig. 7.

Examples of interactions seen in our observations. Clockwise from top left: several participants interact together; two immersants dance together, one in VR; immersants hold each other as they interact; a headset-wearer high-fives another immersant.

(© John Desnoyers-Stewart, 2020)

Connecting by touching hands seemed to prompt immersants to playfully explore larger possibilities of tactile interaction: We observed people hugging, trying to touch their legs together, reaching out to touch a stranger’s head or inviting someone to waltz. For headset-wearers, the neutral appearance of the virtual bodies around them appeared to reduce their social inhibition. Subsequently, the interaction between a headset-wearer and others around them seemed to occur across the social norms of two distinct spaces, as indicated by observations of touch that would not have ordinarily occurred between strangers. In most cases, the headset-wearer initiated contact and the other reciprocated, becoming more active as they become more familiar with VR and their ability to navigate the space around them. These unusual forms of social contact may have enabled interpersonal embodied interactions and formed what Bourriaud calls a social interstice, where connections emerge in ways not otherwise possible outside of the installation.

Conclusion

Body RemiXer explores the potential of inviting intersubjective experiences in both actual and virtual realms by linking interpersonal tactile interaction with ethereal virtual representations. This presents new possibilities for MR to connect people by mediating co-present embodied intercorporeal interactions. We have proposed a way to encourage social connection by visually limiting the perception of the body’s outward appearance while amplifying its movement and connection to others. We continue to build upon this knowledge, exploring the phenomenology of virtually transcending one’s body to connect with others through in-lab studies and continued observations of installations in the wild.