Immersion. Between simulation and re-entanglement

When writing about immersion in the context of the (digital) arts, the focus tends to be on complex and expensive assemblages that are often cutting-edge in terms of technology, for example 360° cinema or VR.¹ On the production side, these can entail specially equipped 360° studios, panoramic cameras, ambisonic or binaural sound equipment, motion trackers, sensors, and the like for recording, as well as powerful computing facilities for modelling, editing, and rendering in post-production. On the reception side, these assemblages include head-mounted displays (“goggles”), large screens, hand-held controllers, different devices for haptic feedback to be strapped on the body, 360° projection, multichannel surround audio, sensors, etc. All in all, it is a dauntingly complicated, quite unnatural situation that requires specialist knowledge and frequently works only when closely tended to by technicians. And, indeed, we often encounter immersive works as a technical assemblage. Or fail to encounter them when the installation is out of order right when we want to immerse ourselves.

This technology-focused view of immersion and the linear progress story it tells (“from illusion to immersion,” as Oliver Grau puts it²) seriously limits our thinking about immersion. First, by presenting it as an exception, the brief moments when we enter the projection space or don the goggles and it actually works. This immediately begs the question: exception to what? Detachment? This makes no sense. We are not detached from the world unless we enter such high-tech spaces. Thus, I want to turn the perspective around. Building on classic positions in media theory, I see immersion as the normal state of perception and detachment as a specific, historically and culturally situated mode of perception, created by “old media” that formed what Marshall McLuhan called the “Gutenberg Galaxy.”³ Despite the fact that the end of the Gutenberg Galaxy was announced more than half a century ago, we are still in the process of leaving it, with many of our Western intuitions and common-sense assumptions still informed by it. Deep cultural change is a slow, multi-generational process.

Second, if we consider immersion a normal state of perception, then we need to think more critically about what happens when creators use complex technology to build “immersive environments.” Importantly, it cannot be the simple fact that they create immersive experiences, as these already exist without the technology. Rather, I contend that they create aesthetics that articulates the particularly contemporary character of immersion, shaped by the condition of digitization and climate breakdown; this aesthetics has consequences.

Here, I identify two main directions for contemporary immersion aesthetics: one focusing on creating simulations (in the sense of Debord’s spectacle or Baudrillard’s hyperreality), and the other creating experiences of re-entanglement with the wider world, each with distinct technological and ecological politics attached.⁴

In the following, to substantiate the first thesis, I present a media-historical sketch of the genesis and current crises of a detached mode of perception, with the world in front of us. Then, I will look at the “simulation” and “re-entanglement”, as two aesthetics re-articulating immersion under contemporary conditions. The whole text is intended as a short and pointed provocation to the technology-centered view of immersion.

How the World Moved to the Front

If we treat immersion as the normal state of perception, I want to discuss the first thesis: the idea that the world is in front of us is an unnatural one in many ways, with a distinct cultural history. For one, this notion clearly goes against everyday human experience. All senses but one indicate that the world is all around us, and already being in the world constitutes an existential precondition for sensing anything at all. Only the visual sense, if separated from all other senses, can create the illusion of reality being in front of us, and that we live in distance to the world rather than in connection with it, and that detachment is a requirement of perception and knowledge. Yet, this separation of the senses seems such an unnatural form of perception that an entire techno-cultural environment was necessary to create it. Aesthetically, the central perspective, first articulated by the architect Filippo Brunelleschi around 1420, formed the key element of this environment. It aimed to represent the visible world by transferring three-dimensional objects onto a two-dimensional surface with high visual fidelity. The claim to veracity was established in a famous experiment, which the art historian Samuel Edgerton describes as follows:

After painting the picture [of the San Giovanni Baptistery in Florence] Brunelleschi drilled a small hole through the panel and then had his viewer look through this hole from the backside of the panel. In the other hand, the viewer was to hold a mirror in front of the picture, reflecting the image as seen through this hole.⁵

Holding the mirror in the hand, the viewer could not only see the mirror image of the painting, but by moving the mirror away, they could see the Baptistery itself. In this way, the direct correspondence was established between the external reality as the immobilized eye could see it and the representation that the painting showed. One might say that the mirror constituted the first in a long line of technical apparatuses to produce images that would represent the world as it could be seen “naturally.” Thus, others, absent in time and/or space, could see it as well. This direct correspondence between the image and external reality, between the sign and the signified, is called its “indexicality.”⁶

By calling the central perspective a key element of a techno-cultural environment, I want to highlight that aesthetics does not simply mean a passive representation of sensory input or social relations. Rather, it is an active force in making the world accessible in a specific way by shaping our perception: of ourselves, the world, and the relations in-between. We can say the same about the printing press, which helped to make reading and writing on paper the dominant way of knowing and organizing the world. As many media scholars have pointed out, printed text also moves the world in front of us, by translating all human experience into a linear visual experience: letters on a page.⁷ Moreover, it requires a certain distance from the world, in order to decode the otherwise meaningless alphabet letters in one’s head. Emerging more or less at the same time, it represented more or less the experiences of the same group of people, the rising merchant class in cities.⁸ The perspective and the printing press transformed the Western subjectivation, namely, the pattern of experience of oneself and one’s relationship to the word.⁹ Reading a text and looking at a painting had, and still have, a lot in common. Both take place individually, in silence, and, ideally, without any other sensory input or bodily movement. Hence, the modern museum and the modern library are very similar institutions.¹⁰ They formed a new type of the subject: the autonomous individual. This subject became dominant, not because of some technological determinism, but because the new way of seeing and being turned out to be “superior” in organizing the world, both in terms of commerce and state-craft, as expressed in ever more complex forms of production, trading, and consumption as well as in ever-expanding state and private bureaucracies.¹¹

This subject not only believed that the world was in front of them but also that more or less passive objects populated this world and could be rearranged freely according to utility. The relationship to this world that the subject found in front of themselves was one of distance. This distance proved crucial both to the scientific method separating the observer from the observed (enabling “objective knowledge”) and to the demands on the subject: rationality and self-restraint.¹² Calling this way of organizing and being in the world “superior” does not imply a value judgment. Rather, it is an observation of its competitive success, slowly and often violently pushing all other ways of knowing and being to the margins.¹³ However, in many ways, this approach has become a victim of its own dominance. It created forms of bio-socio-technical complexity that its social forms (including its aesthetical forms) can no longer contain, increasingly undermining its own stability, to the point where it has become entirely uncontroversial to say that we are facing a “poly-crisis” and numerous threats to “sustainability.” There is a widespread feeling of “information overload” which does not primarily indicate a rise in the quantity of information, something notoriously difficult to measure.¹⁴ Rather, mentally and institutionally, it indicates that the social forms through which we process this information are not up to the task, despite the devotion of entire industries.

We Are Forced Back into the World

Two of the main drivers of the crisis of the modern mode of perception are digitization and climate change. Among many other things, they also pose a profound challenge to the notion that the world is in front of us and that the best way to understand it is to distance oneself from it. I remember when my mother, then in her mid-70s, started to use the internet through an iPad. Many of the technical difficulties at that point had been solved by Apple designers. Yet, she found it hard to use at first. One of the sources of real confusion was that there was no beginning. Opening a web browser, one was always already somewhere, and there was no point from which one could get an overview. It took me a while to understand it, but this was a very astute observation of the breakdown of classic Western aesthetics. There was no longer any order of foreground, middleground, and background; rather, there was only a stochastic string of specific positions, each furnishing its own horizon of visibility, and there was no way of ever distancing oneself to the degree that an overview could be achieved. All forms of knowing immediately revealed themselves as situated, namely, localized and partial.¹⁵ Long held up as the view of the master, the single perspective became just one of many fragments and new ways of relating these fragments were necessary. The experience resembled moving from the commanding heights overlooking the action, right into the middle of a confusing skirmish.¹⁶

On the other hand, the experience of climate change seriously undermines the notion that the world is populated by passive objects, whose only value lies in their availability for human use and exploitation. The perspectival image, which freezes the world for human accounting, everything at its place, waiting to be examined by the discerning eye, much like a merchant examines his wares,¹⁷ can no longer contain a world made up of ever-changing relations, by a continuously expanding range and number of actors. Human agency is only one form of agency, and as we learn with each new flood, heatwave, forest fire, and drought, perhaps not the most powerful one, even in the Anthropocene.

Obviously, from an art historical viewpoint, the crisis of classic Western aesthetics is not new; indeed, it is about 150 years old, rearticulated by every generation of artists from Paul Cézanne (1839–1906) to, at least, Andy Warhol (1928–1987). Nevertheless, by separating aesthetics into its own field (art in the bourgeois framing), the articulation of this crisis had relatively little impact outside the arts, particularly as the torch of perspectival representation of the world was passed to technical means of perception, primarily cameras. How much we still depend on this form of representation manifests itself in contemporary discussions regarding generative AI and deep fakes. Scholars see them as destroying the “indexicality” of the image and, consequently, any stable reference to “reality.” Perhaps the fact that we can easily see that such fears are both simultaneously justified and ridiculous testifies to our historical moment of transition.¹⁸

Thus, even Western culture, so deeply committed to the modern ordering of the world in the Foucauldian sense, is coming to the realization that the old forms of representing and relating to the world no longer prove useful. The world is not a set of passive objects in front of us, but an endless chain of intra-action, constituting and reconstituting all the elements involved.¹⁹ Achieved implicitly or explicitly, this realization contributes to the panic that currently grips Western culture.

Similarly, climate change brings an end to the views that defined the Gutenberg Galaxy. In Earth System science, the world is not out there, ready-at-hand for humans to interact with it.²⁰ Rather, humans are only one of many actors whose very actions constitute the system and each other.²¹ There is no outside, but only specific positions within a complex, dynamic system, and multiple relationships of exchange and interdependency characterize these positions. There is no place for detachment. And the complexity cannot be reduced to a few mechanical relationships of stable parts. Instead, everything is constituted through intra-action.²²

Against this background, the question of the aesthetics of immersion takes on a great urgency, because different aesthetics make different aspects of two dynamics causing the terminal crises of the modern world – digitization and climate change – accessible to our perception. They shape how we can experience the world and our changing place within it. And it is here where aesthetics and politics become inseparable. For the sake of simplicity, I will sketch here only two cases. First, when immersion leads to an aesthetics of simulation through which one reality displaces the other. Second, when immersion leads to an aesthetics of entanglement, making visible aspects of the already existing entanglements that might be imperceivable.

Aesthetics and Politics of Simulation

A powerful current runs through contemporary techno-culture. It believes in computation’s near-unlimited capacities to make informational simulation at least equivalent to the embodied experience. This reflects, at least in part, the still dominant cybernetic thinking which centers around “information.” Information flows define a system, and as long as the information flows remain unaffected, the material substrate can be changed without consequences. Let us take the classic, most basic example of a (first order) cybernetic system: a heater with a thermostat set to keep the room temperature at a certain level. If the room temperature falls below the lower threshold, the thermostat will relay this information to the heating system which will produce heat until the information is received that the room temperature has passed the upper threshold, and then it will stop. From a cybernetic perspective, it is absolutely irrelevant whether a human or machine performs the function of the thermostat (measuring temperature and relaying information).²³ Only measuring and relaying information matters. This basic idea – if one can only control the information flows in sufficient detail, then the material basis of reality becomes irrelevan – underlies this persistent push toward immersion as simulation which refuses to die despite repeated failure. Its most recent iteration has been Mark Zuckerberg’s vision for a metaverse where increased computational capacities would enable the simulation of human interaction, “where you are in the experience, not just looking at it.”²⁴ So far, this vision on which Meta has spent billions of dollars failed, but we can easily see how this can be read as a technical failure – information was not relayed in sufficient quantities and with the necessary fidelity – rather than a conceptual failure stemming from ignoring the ontological gap between matter and information. The problem is deemed a technical one with a technical solution (next version!). And Facebook is not alone. Also Apple is pursuing this approach.

If we assume that immersion as a mode of perception does not depend on VR as a technology, then we can see how the notion of immersion as simulation, so central to a tech-centered vision of the world, reappears in unexpected places. The vision of exploration and colonization of space can also be considered in this light of faith in simulation. Terraforming Mars is nothing other than the plan to rearrange the substrate of life in a way that the necessary information flows to coordinate an ecological system can be recreated: a kind of material virtuality. The belief in the capacity to simulate experience runs so deep that the technocratic elite has seriously discussed the question of whether we are already living inside a computer simulation over the last two decades.²⁵

In a way, we can treat it as an extreme version of the Cartesian mind/body split, in which the body completely disappears. And like the mind/body split, the faith in simulation has ugly politics. For one, it is an obvious and extreme concentration of power in the hands of those who create the simulation. No wonder this seems so attractive to those who see themselves in that role. Be that in the metaverse or the Martian bubble, inside the simulation both the means of production and the means of existence will be owned by capital, triggering a new round of enclosure and accumulation (somehow, the costs of producing air on Mars have to be recouped from the consumers…). However, not only the promise of a power grab creates the allure of this position. It is also a way of dealing with climate change. In a form of extreme anthropocentrism, or, perhaps more accurately, silico-centrism, climate change appears only as a relevant problem, or an “existential risk,” to the degree that it prevents the development of science and the forces of production necessary to create the material basis for complete simulation. As long as this is not the case, colonizing outer space and/or developing sufficient computing infrastructures (ideally also in outer space) will achieve enough simulated happiness in the future to justify the real-world suffering in the present. This is the explicit position of long-termism.²⁶

The dominant trajectory of technology development, particularly of “immersive tech,” leads in this direction. It is the aesthetics of higher resolution, greater speeds, denser sensory input, and machine learning with large language models, of approaching and happily crossing the “uncanny valley” in the never-ending march toward the future. The content of the simulation, or its visual appearance, does not really matter, as long as everything relevant happens inside of it. Because outside, there is little more than Baudrillard’s famous “desert of the real”²⁷ where the negative externalities of the simulation itself accelerate the process of desertification.²⁸ Here, the medium really is the message.

Immersion as Re-Entanglement

Another strand in the development of immersion radically opposes this notion of simulation. It aims to foster a renewed connection between people and their more-than-human environment. Many examples within this strand have a resolutely anti-technological (“get away from the screens”) and deeply humanistic nature, meaning they put the human experience at the center of the world.²⁹ The most superficial articulations of this tendency are mindfulness programs, which aim to resensitize people to the small wonders of the everyday. There exists a myriad of different expressions of this, not all superficial, some more esoteric, some more political (at times with strong conspiratorial bends), and some focused on objects, people, communities, or different aspects of the natural world (such as apps to identify plants or birds). They all intend to re-establish connections that according to them were lost, for one reason or another. They speak to a particular historical moment, when to many reality feels to be slipping away, but contain deep echoes of the romantic rejection of modernity, favoring authenticity over abstraction, experience over reflection.

I share their rejection of the rush toward simulation. However, their anti-technological and humanistic stance, as well as their focus on the local and the immediate, fail to live up to the historical challenges that digitization and climate change present to us. In different ways, they both help us realize, or force us to realize, that life unfolds on a planetary scale, but, for better or (more often) worse, so does agency, human and otherwise.³⁰ Climate change and the increasingly violent feedback loops that push human cultures toward tipping points into uncharted territory require recognition of actors and forces that operate beyond human scales, periodically reappear in ways not easily accessible to the human senses, and do not bend to the human will.³¹ To make these different scales and types of agency accessible to perception, we cannot rely on human senses alone. Rather, we need to find ways to express these relationships, to better understand them and, through them, ourselves. In many ways, this is the classic task of aesthetics: to make something available to the human senses that otherwise would be impossible to perceive. Today, this means making it perceivable that we are immersed in a world where agencies unfold on non-human scales – too small, too large, too quick, or too slow to perceive without technological aids.

Over the last decades, the arts have produced a rich set of practices that articulate immersion in this way. I cannot give an exhaustive list. Instead, I want to refer to only two bodies of works. I must admit that I have chosen them based on personal preference. One is the series of Electrical Walks by Christina Kubitsch (2004–ongoing), where modified headphones make accessible the electromagnetic spectrum that envelops us all the time and is modulated by natural and, most strongly, technological forces.³² The other works are Ursula Biemann’s Forest Mind and Acoustic Ocean in which she explores, with a wide range of technological and cultural means, the many relations and ways of knowing produced within eco-systems, made up of organic and inorganic elements. What connects the works in question for the present purposes is that they both extend the senses beyond what is “natural” with the explicit aim of bringing us closer and deeper into the world, creating an affective and curious experience of immersion.

I want to end with a quote by Ursula Biemann, in which she refers to indigenous practices of knowing. However, we can also treat it as a call for a new aesthetics of immersion:

… knowledge [is] embedded in the environment where knowing something means becoming part of a field of meaningful relations, social and historical relations, relations with all beings who cohabit the space.³³

What differentiates the aesthetics of immersion as re-entanglement that we need from the indigenous epistemology that Biemann refers to is the scale of the space in which cohabitation takes place. It is no longer local. It is now planetary. Developing aesthetics that allows us to experience this type of immersion in the planetary space of contemporary life constitutes a task of existential importance.