God Bless the Metaverse

Virtual reality is more than chasing phantoms in the pod.

It’s now obvious that the information revolution has taken a dark turn. Prophets were sounding the alarm decades ago, but only lately has the gloom infected us average digital citizens. It’s articulated differently, based on political temperament and degree of technical savvy, but the story is always the same.

It’s Mike Mulligan and His Steam Shovel, the gritty reboot. We’ve climbed inside our machine and dug ourselves into a giant hole, and now we’re stuck. Also the machine has ejected us from the driver’s seat and pushed us into a corner and is now bearing down on us, clicking its serrated jaws.

Are we fated to be fodder, or can we once more assume our position as operator? Can we redirect our technologies toward human ends, in service of authentic connection and organic community? In the words of Ivan Illich, can we shape “tools for conviviality?” Can we catechize the bots?

Not all emerging technologies are conducive to human flourishing. Opinions differ as to which are the best candidates for baptism. For many, 3D printing promises a return to a culture of neighborhood workshops. Others see crypto as key to empowering communities each to pursue its vision of the good. A few argue that the ecosystem of biosensors will enable a revival of small-scale agriculture.

But when the subject of convivial tools is discussed around the campfire, there’s one emerging technology that never finds a champion. Instead, virtual reality is typically invoked as a warning, a cautionary tale about where we’ll end up if present trends continue: huddled in our pods, helpless, mesmerized by phantoms while society decays into tyranny, anarchy, or both.

Several years ago, as the virtual-reality hype cycle was gathering steam, one viral image set the dystopian narrative. It’s a photo of an audience of conventioneers, eyes and ears encased in identical headsets, mouths stupidly agape, while down the aisle trots Mark Zuckerberg, smirking as if he’s just picked their pockets. And let’s face it: even the most saintly among us can’t encounter a person stumbling around in a VR rig without fighting an uncontrollable urge to mess with them.

I spent five years designing for VR, including stints at Google Daydream and HTC Vive. As you’d expect, tech-optimism was the dominant flavor in those environments. A nonstop parade of new sensors, displays and SDKs inspired ever more totalizing experiments in full sensory immersion. Still, there was a palpable undertone of ambivalence. We discussed the real and potential dangers of the emerging medium, how it might enable new forms of addiction, abuse, and exploitation. We used the word “creepy” to demarcate the line between good innovation and bad innovation.  But “creepy” is an inexact term, and virtual boundaries tend to shift.

Today people are starting to write about the “Metaverse” – a term for any cohesive virtual-reality universe – not as a metaphor, but as a real place you can inhabit. It’s a place largely ruled by Meta: the company’s flagship Quest accounts for about eighty percent of the VR headset market. The company-formerly-known-as-Facebook has spent the last several years under a cloud, suspected of playing fast-and-loose with our collective data. To the extent we migrate our work, our play, and our shopping to the Metaverse, we open ourselves up to being quantified at a much higher resolution than ever before.

So, should we reject virtual reality as the work of the devil, an invention that inevitably tends toward atomization and technocratic control? To do so would be shortsighted. As an artform, and as a practical toolset, VR presents some unique opportunities to imagine and construct a more human-centered society. 

____

Number one: it’s anti-zeitgeist. Oculus and its kin are slowly gaining market traction, but sector growth has been nothing like what insiders were expecting eight or nine years ago, when the first generation of consumer-grade platforms was on the drawing board. Breakthroughs in 3D-graphics processing, and a rising generation that prioritized experiences over stuff, seemed to suggest that the future would be all about full sensory immersion.

We developers spoke of this as a boon. The unified, focused experience of VR would be a respite from the cacophony of life in the smartphone era, with its endless alerts, notifications, and social media scrolls.

Then the platforms launched and failed to take off. Analysts were summoned. They explained that potential users were nervous to adopt a technology that blocked them from their smartphones. We developers responded by scrambling to incorporate alerts, notifications, and social-media scrolls into the VR experience. 

But our initial instinct was right, spiritually if not commercially. It’s become a platitude to note that smartphones have ushered us into an age of distraction. As screens have multiplied, our attention has divided. Forced to compete for scraps, most visual mediums have compressed, becoming more obvious, more shrill and hectoring.

Virtual reality, by its nature, can’t function as one more flashing billboard in our global Times Square. It demands the user’s full attention – visual, auditory, and kinesthetic. If we let it, it can retrain our attention. A thoughtfully designed VR experience can knit our unraveled sensoriums back together and encourage us to be more fully present in the moment.

What sorts of stories will naturally flow from such an all-encompassing medium? As the ecosystem was forming, developer Chris Milk advanced the thesis that VR is “a machine for empathy.” Immersive storytelling would foster mutual understanding and help usher in an age of global brotherhood. Inspired by this notion, and buoyed by grant money, independent producers began pumping out Socially Significant Experiences. In most cases, this meant casting the user in the role of designated victim, to give him a first-hand taste of oppression. The results were circulated at festivals and talked up by tastemakers, but all the promotion couldn’t mask the truth: these attempts were all misfires, both as art and as activism.

It turns out that oppression is unpleasant. Chris Milk was wrong: empathy flows better in media that are cooler. Words can inspire empathy. Photos can inspire empathy. Full bodily immersion only inspires the urge to run away. It also magnifies every directorial choice. What works as effective messaging on a flat screen comes across as unbearably propagandistic in VR.

Virtual reality lends itself best to experiences that seduce rather than assault: environments that invite exploration, stories that unfold at a pace set by the player. Filmmaker David Lynch talks about the concept of “room to dream,” when a story hints at possibilities and allows the viewer’s imagination to fill in the blanks. Today’s moviemakers, too anxious to ensure that audiences connect the dots, rarely leave much room to dream. But open-ended storytelling, storytelling that leaves room for imagination, is natural for VR. It’s a platform for distributed dreaming.

Commercial developers have been slow to grasp this truth, but independent developers instinctively understand it. One such indie project is A Piece of the Universe, a “virtual diorama” by a 3D artist known as Naam. When we boot it up, we find ourselves in a low-poly version of the sort of liminal space beloved by Lynch. A desolate stretch of highway. A fenced-in area alongside the road. Inside, a travel-trailer surrounded by piles of detritus. There’s no set-up, no obvious game-goals, but you can move through the space and interact with just about every object. As you explore, you make associations and sometimes trigger surprising events. Stories emerge, but it’s unclear whether they’re emerging from you, from the environment, or from the interplay between the two.

Another indie project in the same vein, one that has received a bit more attention, is Lucas Rizzotto’s Where Thoughts Go. Again you’re drifting through a surreal low-poly landscape, but here your goal is more defined: to find floating orbs and tap them to release audio snippets. Each is a message left there by another player: a private hope or fear or confession; and you’re invited to leave your own musings for other players to discover. It’s like a spatial, navigable version of the collective unconscious.

Eternal Notre Dame, released this past January, is a virtual tour based on high-res scans made before the 2019 fire. It’s appropriate that the cathedral has been restored first in VR. The building is in a sense a forerunner of VR: a self-contained, fully immersive world that invites contemplation. The work of Naam and Rizzotto may not be informed by the same cosmic vision, but they are working in the same tradition as the anonymous designers of the twelfth century.

Still, if VR excels as a medium projecting ethereal worlds, its most unique function is its ability to reflect the player back to himself.

____

Jaron Lanier’s company marketed the first commercial VR headset way back in 1985. Today he writes books with names like You Are Not a Gadget and deplores the Meta-stacization of the industry he helped found. The new generation of developers misses the point, Lanier believes. The natural subject-matter of VR is not fantastical environments, AI characters or immersive movies; the natural subject-matter is you, the player. In Dawn of the New Everything he writes: “Your center of experience persists even after the body changes and the rest of the world changes. Virtual reality peels away phenomena and reveals that consciousness remains and is real. Virtual reality is the technology that exposes you to yourself.” 

One of the more successful VR games is Superhot. It’s a shooter with a difference: the speed of the bullets and fists coming at you is precisely correlated to the speed of your own reactions. The game is less about quick reflexes than precisely-controlled actions. The more you play, the more you become aware of your every bodily movement.

A medium with the word “reality” in its name will constantly be hungry for greater fidelity: wider visual field, more precise spatial audio. Pioneering developers are always and forever soldering new input mechanisms onto existing systems. 

My first VR design gig was at a start-up called Eyefluence (motto: transforming intent into action through the eyes). There I worked on interactive experiences for which the player’s gaze was the primary input. The theory was: the eyes are the fastest moving organ in the human body, and the organ most directly connected to the brain. Ergo, eye-based interaction should be the most intuitive and rapid way to control an interface. 

When our experiments succeeded, the results felt something like magic. It was as if the system were reading your mind, serving up an answer before you’d finished formulating the question. With practice, you could learn to navigate complex interfaces with the eyes alone, to make plane reservations or place Amazon orders at the speed of thought.

But too often our experiments illuminated the limits of conscious control. The eyes are only partly obedient to our will; mostly our gaze follows prompts that lie far below the level of awareness. Strapped into our gaze-based experiences, testers would inadvertently trigger cascading chain-reactions in their virtual environments. They’d experience a loss of control, even panic. Sometimes they would claw off their headsets in desperation. 

There’s a value to illuminating our bodies’ reflexive response systems. In the context of a training simulation, gaze-tracking can reveal the moment my attention wandered, or when I noted a telling detail (Inveris has leveraged it for VR-based police and first-responder training). 

Other experimental simulations have involved outfitting testers with galvanic skin response monitors or brainwave trackers to learn more about how subconscious processes inform our actions. Some of these cross the line into “creepy”. But ultimately this research gives us a fuller picture of the human condition. It can help us train our habits and instincts. It can help dispel Enlightenment-era myths about rational agency. It can make us a bit more humble.

It can also make us more transparent to others, for better or for worse. So far we’ve been discussing solo experiences, but Meta wouldn’t be betting so hard on the Quest if it didn’t believe the future of VR was primarily social. What does it mean when a communication platform can transmit not just the love sonnet you wrote, but the prickly heat on the back of your neck while you’re reciting it?

____

Of course, Meta wants to position its own Horizon Home as the default VR social platform. So far, its Disneyfied take on the metaverse has bought it limited love. Other social platforms like Engage and AltSpace have had some success as venues for virtual meetups; good clean fun if you want to attend a convention dressed as a giant playskool figure.

Only one immersive social platform has inspired true devotion: the trippiest platform of them all, VRChat. Imagine exchanging cocktail-party banter with a xenomorph and an anthropomorphic horse, while overhead a fluttering anime angel interjects outbursts in Japanese. All on a rooftop garden, in a neon cityscape under a blood-red moon. It’s disorienting, it’s cacophonous, and it represents daily life for a growing number of users.

VRChat understands something fundamental that the others miss: you don’t go to the metaverse to hear a lecture about breakthroughs in orthodonture. You don’t go to hang out, to watch movies or even to play games together. You go to perform.

If Lanier is right that VR is the technology that exposes you to yourself, it’s a de-psychologized version of you. In a virtual social environment your words and attitudes are no longer constitutive. Your self presentation, physical position and actions are all that matter. There’s a word for a pattern of behavior that expresses your identity in relation to others: ritual.

VRChat is a laboratory for creating and playing with rituals. It’s an outlet for self-expression and a social forum for many otherwise isolated individuals. True, the styles don’t cohere, and the actions don’t add up to anything. Most of the people you encounter here seem to be performing for nobody in particular; eventually you discover they’re staring into invisible mirrors. Another paradox: the first true virtual society is being built primarily by autistic teenagers.

What could be achieved if people who actually know something about ritual, people who understand how symbolic forms can unify and motivate groups, decide to construct virtual worlds? VR is a place where symbolism is everything. Your uniform is not just an outer cloak that signifies your station – it’s your entire form. A promotion or demotion might mean a change in physical size relative to others. Ceremonies can be enacted across distance: gang initiations, rites of passage, and purification as digital media. This raises the possibility of distributed thick communities.

I hear the objections from those readers who haven’t yet drunk the Kool-Aid. We’re already embedded in communities, and those communities are becoming thinner and thinner, largely thanks to digital media. Our cities are burning, our bridges are crumbling, and no amount of virtual pageantry is going to fix them. 

Fair point. But VR also has many applications in the practical world. It can, in fact, help restore a culture of building.

____

It’s been over a decade since Peter Thiel famously asked “What happened to the future?” In the time since, legions of observers have signed on to the thesis that, despite the constant hype about technical breakthroughs, our culture is in a period of stagnation. All innovation is concentrated in the world of bits and bytes, while the world of atoms languishes: infrastructure crumbling, building projects stalled, healthcare in decline, aging fleets of airplanes crawling along at a measly 500 miles per hour.

All this because reality is hard. Innovation in the digital world is frictionless, and failure is almost cost-free. Innovation in the material world is full of friction: regulations, lawsuits, and logistical snafus. Cost overruns for large office buildings run to thirty-three percent. Failure is costly, and sometimes deadly. Back in the heroic age, you could build a Hoover Dam and factor in a few slip-prone construction workers as collateral damage. But in our increasingly safety-obsessed culture accidents have become intolerable.

Virtual reality might have compounded the problem. If it had achieved the mass-success predicted in the last decade, your employees would now be donning headsets to meet at the virtual office, and there would be no need to renovate the real office you closed down during COVID.

Mass-success did not occur, but something else did. Sensing the potential of VR as a tool for design and planning, certain real-world industries incorporated the technology into their workflow. 

Surgical teams are using it to perform dry-runs of particularly tricky procedures. According to Sabine Girod, a Stanford Medical professor who specializes in reconstruction of complex cranial-facial injuries, virtual rehearsals have brought the average time of these procedures down from six hours to three-and-a-half.

One of the few runaway successes in VR is Gravity Sketch. It’s a tool that makes digital 3D design as viscerally satisfying as sculpting in clay. The app has been adopted as part of the standard toolset in the auto industry. Manufacturing procedures can also be modeled and tested in VR. An environmental engineer with John Deere has informed me that her company models all processes first in VR, to ensure maximum efficiency and ergonomic safety.

VR’s you-are-there quality makes it an ideal medium for evangelizing ambitious projects. One of my favorites was funded by the United Arab Emirates. It’s a tour of the monarchy’s first Mars colony, circa 2117. A virtual guide points out the various features that sustain the colony, explaining how the systems evolved from traditional Bedouin desert-survival techniques. It may not be the most sophisticated VR experience out there, but it exudes a refreshingly wide-eyed cultural optimism. 

____

Does VR deserve the bad rap it’s earned in the “convivial” community? Perhaps. To the extent we employ it as an escape from our lives, an idealized alternative to the real world, it will function to dissolve the bonds between us and make us less substantial. Sure, the same can be said for every mass medium since the printing press. Isn’t Don Quixote in part a warning against virtual world-building?  But the inherently totalizing tendency of the new technology, its ability to counterfeit sensory experience, enhances the danger that we’ll lose ourselves in it.

Whether we like it or not, the future is digital, and “immersive computing” is going to be part of that future. If we can learn to order the medium toward genuine human ends, the same qualities that make it uniquely dangerous might render it uniquely beneficial. Spatial, tangible experiences, interfaces that engage all our limbs and sensory apparatus, can help to re-ground us. When all the fantasy is stripped away, VR is a reminder that we’re embodied creatures in a given world.