An Early Manifesto for Spatial Computing

Spatial computing is a new material. Some materials are more responsive to some design patterns than others. It makes no sense to say, “Skyscrapers should be built out of cotton – if only cotton were strong enough.” It’s a better policy to figure out what design patterns play to the strengths of that material.

The gaming industry will probably do most of the grunt work to sort out which design patterns best fit spatial computing. Spatial computing (an umbrella term for a plethora of hardware, software, and sensor assemblages) submerges the user into an interface. The experience is inherently more dimensional than flat screens. Video games are more graphical, dimensional, and multiplayer than textual interfaces, so there is a strong incentive to figure out how XR can expand on games.

The command line is probably the first digital textual interface. It had one job: manipulate symbols to provide instructions for machines. It was only possible with a keyboard and a screen. Keyboards were ripped completely from typewriters. Screens were the digital analog of the typewriter’s output, analog ink and paper. But the screen turned out to be a much more potent material than its original intent. Decades of innovation and revenue spilled out of it. Today, it is hard to imagine software without screens.

Text interfaces haven’t faired well in the last decade. One can’t help but feel overwhelmed by text, whatever its origins. Social threads, internal documentation, and countless tabs (of countless ad-riddled articles) are all varieties of textual interfaces. Synoptic comprehension and intuitive collaboration have suffered under the current hegemony of flat graphical text interfaces.

Could text become less overwhelming if screens lost their dimensional constraints? Is XR the right material for word processing?

I bet that conventions around text will have to change first. Multi-modal devices and AI augmentation blow apart current conventions about what writing is. Articles may not be written so much as composed into polymorphic shapes that complement whatever material they arrive at.

We’re at a rare moment when several layers of abstraction are up for grabs. Just as we’ve been trained to drive cars, we’ve been trained to operate personal computers. The heuristics are, for the most part, locked in. Mobile phones are much younger than PCs, but most heuristics are also settled for phones. By comparison, XR is uncharted territory.

Even though games will define the heuristics of spatial computing, this is the time to push the paradigms of personal reflection, public works of digital civil engineering, and possibly uncover latent talents in the material of the human body.