X-REALITY
A MANIFESTO FOR MIXED REALITY FUTURES
by
ATOM FUTURES AND INFERSTUDIO
May 2022
All original artwork by INFERSTUDIO.
INTRODUCTION
Consider the degree to which our perception of the world is shaped by digital media. Across social interactions, commerce, work, education, and political engagement; our lives are hyper-mediated by the virtual sites we live with and within.
In the coming years, the impact of digital media on our daily interactions will only grow, immersing us in extended realities (XR) that render the distinction between the virtual and the real increasingly redundant. With XR, digital interfaces and experiences will leave the screen and enter the physical spaces around us, becoming part of the fabric of life.
Reflecting on those XR futures and where we are now, at the early stages of the Metaverse and Web3 (the decentralised, immersive internet) - characterised by evolving technologies such as blockchain, virtual reality, augmented reality, and artificial intelligence – those of us involved in its creation, including brands, technologists, designers, artists, strategists, and investors, must be guided by certain principles to ensure we build XR futures that are equal, safe, inclusive, sustainable, open and transparent.
This manifesto considers how XR builders can set out on that path in the upcoming Web3 era. Part One presents general principles for the future of XR infrastructure. Part Two presents three speculative scenarios about XR use cases and specific principles for the design of XR interfaces and experiences.
PART ONE
PRINCIPLES: DESIGNING INFRASTRUCTURE FOR MIXED REALITY
There are many lessons to be learnt from Web2 and previous technological revolutions that can inform our collective thinking for the future. The internet as we know it today, for example, is a powerful medium for information sharing and connection, but is also a place that is vulnerable to, and which facilitates, a plethora of exploits and abuses. Social media is one important example of how biases, imbalances and abuses of power and human rights in the physical world, combined with poor regulation online, have created global issues such as online harassment and disinformation.
To ensure we create XR futures supported by digital civic infrastructures that are resilient to the abuses we see today, we present five general principles to all XR builders:
1.0 INCLUSIVE AND EQUAL
Existing gender and racial disparities in the tech industry, associated sectors, and in society generally impact how emerging technologies are created and purposed, running the risk that XR inherits the biases we see in, for example, social media algorithms and facial recognition technologies, and - more recently - the cryptocurrency and non-fungible token (NFT) spaces. Underrepresented groups including people of colour, women, LGBTQ people, people with disabilities, as well as people from different language and age groups, must be meaningfully included as designers, policy authors and UX testers, among others. Design leadership from people with disabilities should be embraced as a means to expand and improve diverse interfaces through which to use emerging technology (for example voice-to-text, text-to- speech, and different forms of haptic feedback), while seeking out contributions from experts in different fields to ensure that questions of identity, self-expression and culture capture a multitude of perspectives, resulting in a more equal and safe space for all.
2.0 SUSTAINABLE AND CONSCIOUS
Any XR futures will require significant amounts of energy to power, causing concern about potential negative environmental impacts. Many people are already worried about the high energy consumption of technologies such as blockchain and cryptocurrencies that are central to Web3. This context requires XR builders to deploy a sustainable mindset. While we wait for ‘proof of stake’ and other more sustainable blockchain mechanisms and models to take hold, choosing to invest in, develop or use less energy intensive blockchains in the short term is just one of many decisions that can put XR builders on the path to ultimately creating carbon neutral or carbon negative XR systems. Our XR futures must also be environmentally conscious and ethical in their design; there is an important opportunity to increase awareness and mobilise users in XR around real-life problems such as the climate crisis, encouraging positive behavioural change and collective healing in both physical and digital global societies.
3.0 DECENTRALISED AND COMMUNITY FOCUSSED
Decentralised and community focussed infrastructures should form the basis of XR futures. Blockchain, the peer-to-peer building blocks of Web3, is decentralised by design, giving us a new paradigm of increased control over our personal data and digital identities. This contrasts with the existing internet, which is predominantly controlled by a handful of companies with a monopoly on personal data. A decentralised model prioritises individual autonomy and anonymity, and an individual’s relationship with the larger community, another core tenet of Web3. NFT collectives shaped by community values and mission-driven, self-governed Decentralised Autonomous Organisations (DAOs) are already showing us the huge potential that can be unlocked through collaboration and co-creation, values which should also be applied in the building of XR.
4.0 TRANSPARENT AND SAFE
Building trust with existing and future users of XR will require a high standard of transparency and safety, issues which social media and technology companies have grappled with and fallen short on in Web2. There should be transparency about how personal data is collected, stored, used and shared, supported by data protection frameworks that operate on the basis of explicit and informed consent, and which adopt a ‘privacy by design’ approach. Proponents of blockchain technology laud its ability to provide a public ledger of all transactions that is visible to everyone, giving us a model of transparency and explainability that can serve as inspiration. The issue of safety is also a key priority, requiring regulations and effective implementation to protect vulnerable groups, including children. Accountability mechanisms and safeguards against issues such as cyberbullying, discrimination and abuse on any grounds, or for any infringements of policies protecting users’ rights, should also be in place and effective.
5.0 OPEN AND INTERCONNECTED
The future of XR as we see it depends on interconnectedness and interoperability. Siloed approaches to building Web3 and future iterations of XR would only result in closed platforms and economies, creating hierarchies and stifling both the movement of users and the utility of their digital assets. An open approach characterised by open-source protocols and infrastructures on the other hand would give those whose lives are affected by XR a say in how it is developed, and promote network-building across virtual spaces and cultures. It would also allow for an integrated financial and information system that enables the transfer of assets and information across different platforms. Achieving that level of interoperability would require that XR builders from across all sectors adopt an ethos of openness and interconnection, with the intention of creating digital civic infrastructure that is accessible to all.
PART TWO
SPECULATIONS: ENVISAGING FUTURES FOR MIXED REALITY
As we apply the general principles for XR infrastructure, we must also start to envisage the XR use cases and interfaces we want to see. In this manifesto, we portray and interrogate such visions of the future.
We imagine how mixed reality technology might enable us to better perceive and engage with the world around us, rather than taking us into alternate worlds. Our manifesto pre-supposes two conditions: a screenless world, and near ubiquitous access to high fidelity audio-visual mixed reality (be that through glasses, contact lenses, neural implants, or other).
At the end of each of the three speculations presented here, we set out a series of principles for XR builders who are reimagining and building such technological futures.
SPECULATION ONE: X-IDENTITIES
I organised a work dinner last night. Nine people from my team happened to be in London at the same time, so I booked a private room at a Thai place in Soho. The connection speed there is excellent - so I was confident that no matter what lens version people were wearing, all of our overlays could be full-resolution, with no glitching. It gives me a lot of confidence when I’m getting ready to go out to know that the textures or features that I choose for my overlay are going to map correctly to my body. I feel in complete control of how I’m going to appear.
Last night for example, I used a colour palette for my skin texture taken from the rainforest my organization is currently campaigning to protect. I’m personally trying to raise awareness about ecological biodiversity, so I used as an augmentation around my eyes this pattern from a lichen that is native to the region. Someone asked me about it on the underground - and I got a bit of public awareness raising done, talking about our campaign.
Another lady heard us chatting and actually joined the conversation - pretty incredible for London and actually sort of hilarious because (as she explained to me) she uses a filter in public spaces that colour codes people’s overlays based upon their likely covid-transmission risk. I was deemed low-risk apparently - so I already looked green to her.
It’s an unofficial workplace dress-code to do some sort of augmentation around your lips. This helps out our CFO who is deaf and lip-reads. She prefers lip-reading to using real-time transcription - and I know what she means. Transcribed sentences generally appear above the speaker’s head, so you never feel like you’re making proper eye contact. I turn on transcription if I’m having trouble understanding someone, but I try not to make it too obvious through my gaze; many of my colleagues are non-native speakers who are proud of their ability to socialise without a translation service. That’s their preference, which I respect, but I personally appreciate real-time social support services.
Last night for instance, I dialled in a coach who helps me navigate social anxiety. I shared my view with her and, watching remotely, she gave real-time advice about conversation starters, or how to use environmental cues. I wouldn’t use that kind of service on a date (I’d find that too intrusive) but it’s helpful in big, noisy environments where I can get overwhelmed.
For example, last night, I was sitting opposite our boss. Like most people, she carries around with her a projected environment; yesterday it was a field of wildflowers from an at-risk ecosystem our organisation is working with. During dinner, the flowers mingled with a projected wall mural that our CFO was sitting in front of, and some kelp from the underwater environment that our boss’s little daughter had around herself.
But then our boss gave a short presentation about the mission of our organization. The lighting in our views all adapted to show her spotlit as if on a stage. I didn’t notice that as she talked, everyone else was gradually adding those same kinds of wildflowers to their own environments.
If my coach hadn’t prompted - I would have completely missed it and looked like I wasn’t listening to the presentation at all.
One-on-one, I actually pride myself on my social skills; I’m good at fluttering my skin matte between different colours to show empathy or enthusiasm - and I’m good at names. I almost never turn on name- tagging because I think people can tell (it’s the quick glance above your head) and hence appreciate it if you remember them. But there’s a time and a place for social support services, and I’m grateful that they’re available to me.
Of course, when I do live-stream my view, a flag is automatically attached to my overlay that says I’m sharing it to someone outside the room.
SPECULATION ONE: X-IDENTITIES
I organised a work dinner last night. Nine people from my team happened to be in London at the same time, so I booked a private room at a Thai place in Soho. The connection speed there is excellent - so I was confident that no matter what lens version people were wearing, all of our overlays could be full-resolution, with no glitching. It gives me a lot of confidence when I’m getting ready to go out to know that the textures or features that I choose for my overlay are going to map correctly to my body. I feel in complete control of how I’m going to appear.
Last night for example, I used a color palette for my skin texture
taken from the rainforest my organization is currently campaigning to protect. I’m personally trying to raise awareness about ecological biodiversity, so I used as an augmentation around my eyes this pattern from a lichen that is native to the region. Someone asked me about it on the underground - and I got a bit of public awareness raising done, talking about our campaign.
Another lady heard us chatting and actually joined the conversation - pretty incredible for London and actually sort of hilarious because (as she explained to me) she uses a filter in public spaces that color codes people’s overlays based upon their likely covid-transmission risk. I was deemed low-risk apparently - so I already looked green to her.
It’s an unofficial workplace dress-code to do some sort of augmentation around your lips. This helps out our CFO who is deaf and lip-reads. She prefers lip-reading to using real-time transcription - and I know what she means. Transcribed sentences generally appear above the speaker’s head, so you never feel like you’re making proper eye contact. I turn on transcription if I’m having trouble understanding someone, but I try not to make it too obvious through my gaze; many of my colleagues are non-native speakers who are proud of their ability to socialise without a translation service. That’s their preference, which I respect, but I personally appreciate real-time social support services.
Last night for instance, I dialled in a coach who helps me navigate social anxiety. I shared my view with her and, watching remotely, she gave real-time advice about conversation starters, or how to use environmental cues. I wouldn’t use that kind of service on a date (I’d find that too intrusive) but it’s helpful in big, noisy environments where I can get overwhelmed.
SPECULATION ONE: SPECIFIC PRINCIPLES
EXTENDING PERSONAL IDENTITIES
This first speculation introduces the concept of personal mixed reality overlays. Like items of clothing, these overlays will be seamlessly mapped to the faces and bodies of users and rendered visible through XR devices like glasses or contact lenses. Users could craft their overlays as carefully as they do their outfits, and the choices they make about colours, textures, and symbols would act as social signifiers just like clothing, hairstyles, and make-up currently do. The difference however is that these new forms of self-expression could be dynamic, shifting within seconds to match topics of conversation, emotions, or attitudes.
REVEALING HIDDEN REALITIES
Crafting a mixed reality overlay is a means to extend and give agency to a personal brand. But they may not remain entirely within users’ personal control. Users may adjust their XR views for additional information - like the fellow traveller our protagonist encounters on the train, who colour- coded people’s overlays with an algorithmic assessment of likely COVID risk. This is an example of the power of XR to make visible realities that are undeniably real, but otherwise difficult to see. An XR future will come with the possibility not only of augmenting one’s own appearance, but the appearance of others around you. What does it mean when we cannot only filter our own identities, but also those of others based on public data streams, or even just personal preference? XR builders in the future will need to be careful to consider privacy, consent and the potential for XR overlays to entrench media echo-chambers.
ENHANCING ACCESSIBILITY
Speculation One also introduces the potential of XR to support users’ diverse physical and social needs. This could take the form of new behavioural conventions - like using a social coaching service to provide personalised real-time support during interactions, in a manner invisible to others around you. If XR assistive technologies exceeded current offerings in their ease-of-use and availability, then an XR world could be easily more inclusive for people with disabilities or forms of neurodivergence than present day society.
COLLECTING DATA TRANSPARENTLY
Speculation One highlights the need for conventions that balance the accessibility needs of users with the right of people around them to know when and how their data is being collected. Our protagonist for instance live-streams everything they see to a third-party ‘coach’ who helps with their society anxiety - but when they do this, a security flag is automatically attached to their overlay. This is the XR equivalent of the red ‘recording’ button on a video call, which everyone can see.
INNOVATING SOCIAL CONVENTIONS
This first speculation hints at the multitude of new social conventions that ubiquitous XR could inspire. Our protagonist ‘flutters’ their XR overlay as an indication of empathy or enthusiasm. Everyone at the dinner table live edits their own XR overlays to show engagement with the boss’s presentation. Like emojis, hashtags, and internet slang, XR communication conventions will define and redefine the field - and the players who get there first will have enormous influence on everyday life in an extended world.
SPECULATION TWO: X-CITIES
Don’t ask me why – but I’m currently training for a marathon. Sunday is when I do my weekly long run. I’m part of a group that meets around London Fields Station – there’s a wall just after you come up out of the underground, where we each place a counter that records our personal best and current live running time. I can see from the counters who has started training before me – and I try to catch them, switching my navigation symbol into a little fox, which shows me the way by darting down the road and sniffing around corners; it’s very cute and motivating to chase after.
The streets around the station can get confusing. It’s a busy area and there’s heaps of meta-data trails about which routes to follow for low-pollution exposure, reduced transmission risks, or construction avoidance. I set my route to minimise obstacles and then filter everything out except for the trails left behind by other runners. The trails look like glowing wires, and people have personalised them in different ways. On my route, there’s a complete 3D scan of an older lady; Running Grandma, I’ve dubbed her in my head, and when I had just started training we ran at the same pace, more or less. I appreciated her ghostly companionship. Other people attach voice-notes; “Hi, I’m Niko” says one trail as I cross it, “I hope you will run with me.” I try to, but he’s currently too fast.
A friend of mine trained for a marathon a few years back. She left voice-notes encouraging other runners throughout parks in central London, indicated by a little blue bird that perches on a lamppost or in a tree. I look out for the bird and play the notes whenever I pass one. Of course there are heaps of anonymous trails too, styled along the street as if someone had run with a dripping can of paint, or left breadcrumbs behind for folks like me to follow.
This week, I needed a bit of a break, so when I left the station, I set my view to historical mode and the whole streetscape suddenly looked and sounded like it may have done before the Vikings came. I was getting into the swing of it when a row
of bright-yellow caution tags caught my attention. Public tags like that, which are forcibly visible to everyone at all times, are necessary but also jarring amidst the grassy landscape. I turned off historical mode, and saw that a construction worker was inspecting drainage pipes from a near-by building.
As I approached, the facade of the building he was looking at seemed to dissolve into a cloud of points, revealing color coded cross-beams, pipes and wires behind. This is the view of the city that he must have every day. To me it was fascinating and beautiful - although also probably something I would normally require special permissions to see.
I make sure to pick up speed around Tower Bridge. There are tourist shops there and on the street outside each store, 3D ‘pop-up’ models of historical figures can suddenly appear right in front of you, advertising old-worldly beer mugs and various types of toffee. I don’t like talking when I’m concentrating on my breathing and even the public safe-phrase to dismiss the pop-ups (“Sorry another time”) is too many words for me.
I actually used to create special filters to use in public spaces while training to block out all third-party content. I still turn off most cookies when I’m out; but there are many commercial environments throughout the city that I like and which are worth tolerating cheaper ads for. For example, on the other side of Tower Bridge is an indoor ski-centre and as you get closer to it, the sky above seems to gradually darken and cloud, and the whistle of a far off wind can be heard. Then flurries of snow start to blow about, and there’s a snowman and a layer of ice across part of the Thames. It’s beautiful and a refreshing change from sticky London in the middle of July.
SPECULATION TWO: SPECIFIC PRINCIPLES
PERSONALISING WAYFINDING
Today, navigating the city is an act of looking down, following a blue dot on a tiny screen. Ubiquitous XR presents the opportunity to bring our gazes back up and into the distance, replacing screen- based navigation with spatial wayfinding. These routing options and navigation assistants can then be presented in personalised, immersive, navigation environments - like the historical grassland our protagonist likes to run in, creating the potential for entirely innovative wayfinding experiences.
RECORDING DATA
Speculation Two also introduces different ways that users could record their presence throughout the city. Our protagonist encounters data-trails from other runners who have left full body scans of themselves, or marked their trails with voicenotes and distributed personal symbols - like leaving messages in a bottle for other people to encounter and dwell alongside.
RESPECTING DATA SOVEREIGNTY
The potential for users to leave XR data-trails in public places also raises the question of data sovereignty - who has rights to this information? Where is it stored? Users may control how much identifying information they make publicly visible and employ community-owned and managed blockchain servers. The design of a safe and respectable XR environment includes the design of options to opt in or out, in varying degrees, to data-collection practices.
PREEMPTING USER SAFETY
In Speculation Two our protagonist’s immersive experience of a historical grassland is interrupted
by publicly visible tags that alert them to a nearby active construction site. In a future where XR environments could be personalised, we will need conventions to make sure everyone sees information necessary to keep them safe - for example, automatic tags around physical hazards that cannot be altered or switched off.
DESIGNING TRANSITIONS
Our vision of positive XR futures is one where the transitions in and out of XR, or from one XR environment to another, are designed to be gentle and seamless, encouraging wonder and curiosity. The journey into XR is as important as the full, eventual experience. When XR is used by retailers, best- practices should respect this and immersion into new XR environments should be gradual rather than shocking, tantalising rather than obtrusive.
SPECULATION THREE: X-EARTH
Today I joined my sister for another virtual house viewing. She and her wife live in California, and they’ve been searching for months for a coastal property with an ‘above sea level’ projection of at least fifteen years. It’s the first thing they check for on a viewing; they turn on the overlay and watch to see if the house will fill up with a projection of water.
I live in London, and I participate in their viewings by overlaying the provided model of the relevant property on top of my own apartment and walking around as much of the virtual space as I can fit into it. I watched the water rise up into the living room for the house in California - and then I turned on the data-overlay for my own apartment, in London. My entire place flooded. Apparently in 2050, my ground-floor flat will be one meter below Thames level.
My sister is also strongly in favour of environmental overlays. She says it ties her day-to-day lifestyle to the wider world. She wants her timber panelling to burn in sympathy with local wildfires as it’s a constant reminder to be prepared and fire-safe; she wants the grout in her bathroom to shine like gold should raw silicon become more valuable on global markets. Most of all, she wants accurate modelling of her house’s carbon footprint.
This is something on which we disagree. Not in terms of intention – I agree it’s important that living spaces physically reflect information about energy usage and emissions – but I don’t think the settings that my sister uses would work for me. She likes white-space, essentially – so she edits the footprint overlay to emit clouds of pink smoke, the more carbon-intensive her consumption patterns are. It caused trouble the last time I went and stayed with her in person, because I found the clouds so beautiful that I ran the air-con in my bedroom that little bit longer, just to fill my space with their pink glow.
Personally, I set aural notifications for each of my appliances, so if I go to turn on the heating when I’ve exceeded my carbon budget for the day, the controller seems to sigh like an irritated parent. My sister finds this especially off-putting – which shows that my settings are actually much more environmentally friendly as she barely touched a light switch on her last visit to London.
I also like to use overlays in my apartment that give me useful information but in a less distracting way. I have pasted overlays onto my fridge that show me how full it is, and where in the world my groceries come from. I’m also tracking how much the fridge is worth on real-time markets - in case I want to sell it and get the optimum price. I have a tracker on my tap to monitor water-use, and when I’m making a new recipe, I attach the steps to the utensils and pans that I’m using for each stage. In this way, everyday data about my life is just there, with me, at home.
SPECULATION THREE: SPECIFIC PRINCIPLES
SOLIDIFYING TELEPRESENCE
Speculation Three opens with an example of the augmented co-presence that physicalised, XR could bring; the protagonist sees - in their own London apartment - a virtual model of the Californian house that their sister is visiting at that exact moment. In a world where so many of us have spent years away from loved ones due to pandemic-related travel restrictions - the possibility of being so casually and intimately together while our bodies are half a world away is likely compelling for many prospective users.
DWELLING WITH DATA
Speculation Three then presents multiple ways that domestic-scale XR can make visible realities that are otherwise difficult to perceive. Environmental data (like how much of a coastal property will be above water in 15 years’ time) could be mapped directly onto the property’s walls - not hidden away in an assessment report. The real-time market value of goods could be displayed above or across them. Users could track energy usage and emissions with patterns, labels, and symbols on their devices, appliances, walls, and doors.
DESIGNING CLIMATE CONSCIOUS BEHAVIOURS
This speculation shows how overlays can be optimised for behaviour-modification. What sort of sensory feedback would make a user restrict their energy consumption - or be more conscious of their water usage? Brands have pioneered ways to problematise aspects of user’s behaviour, appearance, and consumption patterns and then present appealing alternatives. XR presents an entirely new arena in which to exercise this expertise. Climate-conscious and socially-responsible brands could lead the way in innovating XR experiences that encourage environmentally-friendly behaviours.
CONCLUSION: TOWARDS MIXED REALITY FUTURES
Whether brand, technologist, designer, artist, strategist, investor or other, as builders of mixed reality futures we have an opportunity now to start creating digital civic infrastructures which address and overcome known and potential abuses from the outset, and which serve the interests of our global society.
Reflecting on both lessons from the past and on speculations of possible futures, this Manifesto serves to put all builders on a path to equal, safe, inclusive, sustainable, open and transparent future XR realities.
Should you wish to learn more about our Manifesto and how we build mixed reality futures in line with its principles, please contact us at info@atomfutures.com and info@inferstudio.com.
Contributors: Bethany Edgoose, Aura Freeman, Alexandra Ncube, Nathan Su.
All original artwork by INFERSTUDIO.