Some said AR was dead. Today’s technologists disagree. Whether it’s reimagining how we shop, fueling the growth of the creator economy or inching us closer to the edge of the metaverse, today’s mobile AR ecosystem is laying the foundations for new modes of interacting with the world. As part of The Drum’s special Mobile Deep Dive, we dig into the latest goings-on in mobile AR – and scope out what’s on the horizon.
In a 2016 New Yorker article, tech writer Om Malik quipped: “Augmented reality is the ‘boy who cried wolf’ of the post-internet world – it’s long been promised but has rarely been delivered in a satisfying way.” He went on to explore the possible implications of Niantic’s Pokémon Go, which had just rolled out in July 2016. It’s unlikely that Malik could have predicted that the AR mobile game would become lightning in a bottle, attracting some 232 million users.
Half a decade later, and Pokémon Go seems like a half-forgotten fever dream, which some might say supports Malik’s original assessment of AR as an over-hyped but ultimately underwhelming technology. Today’s numbers, however, tell a different story. The global AR market is expected to reach a valuation of more than $340bn by 2028, growing at a compound annual growth rate of a whopping 43.8% between this year and 2028.
Mobile AR has already become a keystone of our daily lives. There are the sometimes lol-able and sometimes all-too-flattering camera filters of Snapchat and Instagram. Then there are the apps whose interfaces offer a digitized glimpse of real infrastructure and geography, from Google Maps to Uber.
Even with AR woven inconspicuously into the fabric of our lives already, many experts argue that perhaps the technological, sociological and economic landscape of our current moment offers more opportunities for innovation in mobile AR than ever before.
Smartify, Google Lens and the AR-aided retail revolution
The growth of AR on mobile has been significantly accelerated by the pandemic and its associated effects. With consumers around the world stuck at home, they consumed media and used their mobile devices more than ever before. In tandem with this shift, the world saw an influx of new mobile-enabled AR shopping and commerce capabilities. A May 2021 Deloitte report indicates that of the 1.5 billion global AR users, 100 million are already using AR to shop.
“We absolutely saw AR take off in major ways during the pandemic, particularly in response to retail and shopping, with virtual try-on experiences from previewing furniture and products in your home with everyday brands like Amazon to virtually trying on luxury fashion labels such as Gucci,” says Dr Helen Papagiannis, a leading AR influencer and author of the book Augmented Human. “Once a ‘nice-to-have’ feature, AR has quickly become an essential technology for retailers.”
A perhaps unexpected player branching into AR-driven commerce is Smartify, the popular AR app that enables users to interact with artwork in galleries and museums in real time. With a database of over 2.5 million pieces, the app can identify artwork in the real world and provide users with more information about the work on their mobile devices. And while the app’s primary objective is to surface information about art to its 2.5 million users, the company is investing in new commerce solutions. “We are doing everything possible to help museums generate vital income after the pandemic,” says founder Anna Lowe. “We are about to launch a museum marketplace so visitors can ‘exit through the digital gift shop’ – there is no better retail opportunity to sell a poster of the Mona Lisa than when the visitor is standing right in front of the Mona Lisa, and only an app can make that happen.”
But perhaps no company is leading the charge on AR retail and commerce trends so much as Google. In December, the tech titan launched two AR beauty tools that enable users to experiment to see what different cosmetics would look like on their own faces – or on a diverse range of models – while shopping for beauty products.
Meanwhile, Google Lens – the tool that enables users to snap a photo of something and find out what it is by dropping it into Google Search for real-time, image-led queries – has debuted new functionality that allows users to shop seamlessly from images or screenshots they take. Last month, the company announced that Google Photos will begin to prompt users with a suggestion to use Lens when screenshots are taken. “Imagine you’re scrolling through a social feed and you see a pair of shoes you like – you can screenshot that picture and then use Lens to identify them and find out where you can buy them,” says Adrian Tout, AR/Google Lens product partnerships lead at Google. Today, Lens can identify over 15bn objects – up from a billion just two years ago. It’s used more than 3bn times a month worldwide.
Tout also notes that AR is now being integrated directly into Google Search on mobile devices. “While Lens lets you search what you see, AR in Search lets you see what you search,” he says. “We began with animals and educational content, and then in late 2020, we began piloting AR for product searches, starting with autos and beauty. While we can’t share specific plans yet, it’s safe to say that we’re evaluating other categories ... all with the larger goal of providing a richer, more visual user experience within Google Search and supporting the broader retail ecosystem.”
Creators will drive the AR of tomorrow
While numerous apps enable users to surface content from the physical world, many experts believe that user-generated content will be critical to unlocking greater potential for AR technology moving forward.
The growth of the creator economy has already been a key force in propelling existing AR mobile platforms. A quick scroll through the endless filters on Snapchat and Instagram Stories immediately evidences the proliferation of user-created content and the ways in which creators are driving the development of AR in the wild.
“Simplifying and democratizing content creation for AR is a key trend to keep an eye out for,” notes Dr Papagiannis. “Snapchat is ... pushing AR forward by empowering creators and brands alike. Lenses within Snapchat can even be thought of as mini apps. We’re seeing the utility and promise of these experiences expanded with new AR capabilities, like Snap’s connected lenses enabling shared real-time interactions whether you’re in the same room or across the globe. This is important because for AR to truly exist as a mass communications medium, it needs to be multi-user and two-way. For the most part, AR has been a single user experience, and that’s beginning to shift in major ways.”
Other major players such as Google are investing deeply in the creator movement. “The biggest area to evolve will be content,” Tout says. “Content is king for visual commerce and it will be no different for 3D/AR.” Tout says there are a number of hurdles that the industry will need to navigate first “before 3D content creation is truly democratized”.
To help advance the creator-fueled growth of AR, Google has introduced a suite of interoperable tools and standards – such as Gitf, an open-source tool for transmitting 3D model data between its creation tools and other mediums. Tout says that the company is helping to scale the ecosystem with open-source tools such as ARCore, Scene Viewer and Model-Viewer – and is actively integrating partner content into its interfaces. “For example, with Model-Viewer, partners can offer a 3D/AR experience on their web property with just one line of code,” he says. “It’s never been easier to make high-quality 3D/AR experiences come to life on the web.”
Marketing in the metaverse
Increasingly, experts see us hurtling ever-closer to a so-called metaverse, where physical and the digital realities are fully melded and a unified semi- or fully-virtual reality takes hold. “The real transformation will come when the challenges with AR headset optics are solved and the notion of an always-on digital overlay of the world becomes a reality,” says Alex Nelson, UX principle of future experiences at the BBC’s research and development branch. “The final remaining question is around who will own the AR metaverse – an always-present virtual world that maps on to the real one. This may seem like science fiction, but being able to present content that is specific to the location and context of the user is deeply compelling to both audiences and those racing to be the big players in AR.”
Nelson helped develop Civilisations AR, an immersive mobile app launched in 2018 that enables users to explore a range of exhibits and interact with famed artifacts in an up-close-and-personal manner.
And Nelson is certainly on to something; the fusion of the physical and digital worlds is a shift that could upend marketing and commerce to an unprecedented degree. In fact, it may not be a far-off reality. “I truly believe we’re on the brink of a new web, a ‘spatial web’, as Gabriel René put it, where 3D will become the standard format,” says Luis Bravo Martins. Martins is chief marketer at KIT-AR, a company aiming to help reduce manufacturing errors and waste on the shop floor with AR and AI solutions that allow manufacturers to visualize correct procedures overlaid on their equipment.
Martin suggests that marketers may face some of the biggest challenges of all in a metaverse. If users are increasingly interacting with the real world through filters, brands will compete to overlay their messages – and their reality – on to the physical world. “That will change outdoor advertising completely, as suddenly, in the same space, [users] can see different and even competing ads. AR will allow [users] to literally recreate this world digitally, so how will your brand help them do that? Bringing our present day web’s content to the metaverse won’t be possible, as we’ll lose screens and add voice and gesture commands.”
The trick will be balancing the objectives of marketing with consumers’ demands for not just data privacy but also humane design and a fundamental respect for autonomy. “Our phones are nearly ubiquitous sensor-packed AR marvels,” says Dr Papagiannis. “The next step is to utilize these sensors to design contextual experiences that are responsive to your physical surroundings while being highly mindful of and without sacrificing privacy.”
In many ways, Martin thinks, marketers will need to bear the brunt of this responsibility – since they’re likely to be the ones shaping this new reality. “We, as marketing professionals ... are the first and foremost barrier between social and economical bliss provided by AR, and dystopia,” he says. “We will be responsible for the first AR experiences ever for most of the world's population ... I see our role changing throughout this decade, adding two more skill sets to our digital, data-centered profiles: keeping updated with emerging technologies, as they’re growing in diversity and are combinatory ... and aligning our brands’ values with the end-consumer’s causes, be them ecological, cultural or philanthropic, but in a way that enables brands to be part of a purposeful customer journey. The CMO of the future will probably be one-third marketer, one-third technologist and one-third activist, but hopefully and above all else, 100% human-centered.”