Anatomy of a deepfake: how Salvador Dalí was brought back to life

Dalí Lives, the latest project from the the Dalí Museum in Florida, is being marketed as an ‘avant-garde experience’ designed to celebrate the 30th anniversary of the surrealist’s death. Goodby Silverstein & Partners were brought on board to commemorate the occasion; the San Franciscan agency decided resurrection was the route to go down.

Roger Baran, creative director at the shop, tells The Drum how he and his team made it happen.

Why did the museum approach GS&P – an ad agency – as opposed to a production shop or another technical business to create this work?

What few people know is that we actually have a long-standing relationship with the Dalí Museum; in fact, this marks the third collaboration between the Dalí and GS&P. Due to our history of successful art projects with the museum, the whole process happened quite organically and naturally.

And don’t see ourselves as solely an ad agency. GS&P is a creative company whose mission is to create experiences that reach millions and even billions,but seem to speak to one of us alone. We call this effect “mass intimacy”, and we pursue not only with ad campaigns. Our motto: is ‘If you know how to do it, don’t do it’.

The truth is that very few shops today have all the skills to bring something like this to life. They are either way too technical or way too creative. It’s a rare and delicate balance to have the necessary talent to create an experience like Dalí Lives because in theory these are very different businesses.

How did you do it?

Our team of creative technologists, led by Nathan Shipley, and GS&P Labs, led by Troy Lumpkin, started researching the technique and brainstorming possible ways to use it. One of the creative teams, Otto Pajunk and Ricardo Matos, had the idea to create a tour of the Dalí given by the painter himself, and suddenly we had the technology to make that possible.

Once we had a concept in place, we defined a few guidelines. We knew we had to respect Dalí’s legacy and his bigger-than-life personality, and that’s no small responsibility. We wanted our AI Dalí to do and say only things the master of surrealism would do and say. So most of the language used by the AI would be sourced from quotes by the man himself (he was quite prolific in that sense, by the way.) The liberties taken would be restricted to allow him to comment on things that the real Dalí could never have said.

The museum began collecting and sharing hundreds of interviews, quotes and existing archival footage from Dalí. GS&P then used these extensive materials to train an AI algorithm to “learn” aspects of Dalí’s face and then looked for an actor with the same general physical characteristics as Dali’s body.

This is deepfake technology: pulling content from thousands of frames of interviews with the artist and overlaying it onto an actor’s face. The historic frames became a digital mask, of sorts, that allowed the actor to appear as Dalí whatever expression he made. We also cast another actor to ensure that the voice matched the countenance.

Dr Hank Hine, the executive director of the Dalí Museum, always says this project was inspired by the artist’s unique sensibility. He said: “I think that the seeds of this project were sown by the artist himself. Dalí was famous for his sense of his own eternal significance. It’s almost like, if he had left instructions for us, this project would have been among them.”

He was right. And we had the duty to be truthful to such an incredible artist.

Did the uncanniness of what you and your team created ever make you feel uncomfortable?

Much the opposite. We were amazed by it. We were proud. We knew bringing Dalí back to life using AI and deepfake could be controversial, but we did it with so much attention to detail, so much passion and so much respect for the artist that we were confident it would be well received.

The few people we showed this to during experimental phases had goose bumps. Some had teary eyes. But in general, the teaser videos really caused an impact on most people who saw it. They weren’t sure if they were looking at archival footage or an actor. More than feeling uncomfortable, we felt incredibly excited with the possibilities of the technology when it’s applied for good – in service of art, for example.

And in a strange way, we were intrigued with the idea of proving the technology can be applied for better ends that it has been used for.

We fully understand the possible ramifications and applications of this technology when it comes to fake news, impacting elections and, of course, involuntary porn. The technology is out there; it’s our responsibility to choose how, and to what ends, we use the technology we invent.

What has this project taught you about the role of the creative agency in culture?

It rather reinforced some of our beliefs: that in advertising we sometimes have the ability to impact culture on a large scale and in ways other professions can’t. And that it’s our duty not only to push products but also to use this power, if you will, to make the world a bit better. We live right on the intersection of culture and technology, and when we use the tools we have for good, we can make amazing things happen.

Today at GS&P we believe creative advertising is more than simply pushing ads and funny jokes. Yes, we always want to entertain our audience, but we know ideas have the power to influence millions sometimes. Advertising isn’t only a result of the environment it is immersed in; it also impacts it and changes it. So, in a way, Dalí Lives is our small contribution to help steer deepfake in the right way.