Last month, I got the opportunity to spend two action packed days at the 2019 Immersive Design Summit in San Francisco. If you’ve followed my writing for any length of time, you’ll have noticed that immersion is a thing I’m into. Here are just three examples: “The rhythms of writing a book in a digital age”, “the 2018 AAM Immersion workshop”, and “On immersion, theatre and museums”.There are more. I think it’s a powerful tool.
One of the things we’re always eager to do at PEM is push the boundaries of what we can do in a museum gallery. And immersion is high on that list, too. And now there’s this book taking shape in my head, so there were plenty of reasons to go. And boy am I glad I did! IDS packed about three and a half days of conference into two lonnnnnnng days, and the quality and variety of speakers was impressive. Rather than try to capture the whole thing, I’ll point you at other summaries of the event, posted here, and here. What I want to do is focus on four people worth following, and four challenges that experience designers face that came up repeatedly at IDS that are germane to everybody making museum visitor experiences.
N.B. If you’re interested in the immersive design scene, the No Proscenium site is a great one-stop shop for all things immersive in North America.
Folks to follow #1: Ida Benedetto
Ida Benedetto was the opening keynote speaker. She gave a talk called “First Steps in Spiritual Trespassing” where she covered immersive experience design, illegal speakeasys, spirit animals, and transformation. It was a good indicator of how diverse the speakers were going to be. After a brief tour of her past work, she took out a rattle and gave the audience a crash course in looking for your spirit animal. And then told us we were going to do just that and then pair up with strangers and talk about the experience. Not your typical “welcome to the conference” warm up!
Transformation requires risk
I was particularly interested in her focus on transformation, since it’s a word that gets bandied about so much in museum mission and vision statements. Ida is interested in how transformation is nurtured, and found in her study of three disparate social rituals — sex parties, funerals, and wilderness treks — that they all share commonalities that allowed them to become transformative. Most importantly, at some level, all three of these activities present an element of risk to participants. Intimacy, and some level of transgression also played their parts, but the element of risk, and strategies for managing it, were central to transformation. Risk and managing it would come up in different terms over and over at the event. It is akin to Johanna Koljonen’s definition of the “interaction alibi” which also got rolled out to an appreciative crowd later that day. Ida’s website, Patterns of Transformation, is a goldmine of ideas on how to design transformative experiences, and a great toolbox for experience designers trying to think about to design human interactions.
“Writing down to the metal” of human experience
For Ida (and many people at the Summit), the real potential of immersive experiences is that they can allow the creator to do what Ida calls “writing down to the metal” of the human experience. Writing down to the metal is derived from the programming term “coding down to the metal”, meaning writing code that accesses the physical features of the hardware it is written for, rather than writing at a level of abstraction that allows code on run on multiple platforms, at the expense of being able to utilize device or platform-specific abilities. Writing down to the metal means accessing those things that are fundamental to our experience of being human. It means designing for people as more than just intellects with eyes, which is still how so many museum experiences are constructed.
And, as has so often happened in really well-thought out presentations, I realized she’d modeled exactly what she was talking about as part of her talk without calling it out as such. The ritual of the conference is itself a situation that carries personal risk with it. Her “stand up if…” activities were the first steps in turning strangers into a community of people with commonalities. The spirit animal activity and one-on-one sharing was a mechanic that allowed us to practice intimacy, and offered opportunity for continued intimacies since “What’s your spirit animal?” became a standard icebreaker for the rest of the summit, and the opener to asking the “So what did it tell you about why you’re here?” question that I can’t imagine asking strangers at a traditional conference.
Challenge #1: Mastering the awkward tango
Fri Forjindam,the Chief Development Officer of the experience design agency Mycotoo, gave a talk called “Disruptive Alchemy for an Immersive Future” that laid bare a lot of the tension that simmered under the surface of the whole event; how to make the kinds of creative expressions the creators wanted while making is sustainable. Her experience in designing large and small projects for commercial and non-profit partners led her to four ingredients that seem to be necessary for truly successful immersive experiences.
1) Bold Blurs – Committing to working in the grey area between fiction and reality, making bold choices and believing in WHY you made those choices.
2) The Awkward Tango – Balancing creative freedom with sustainable business practice?
3) Paradoxes – understanding and committing to solving the seemingly unsolvable contradictions that working in new genres entails. This point was later reinforced by the artistic directors of Strange Bird Immersive in their talk. Their award-winning immersive theatre/escape room hybrid “The Man From Beyond” had serious teething pains, stemming from the paradox of working in mediums that are opposites: escape rooms are audience-driven, theatre is performer-driven. They had to strike the right balance between genres to keep audiences engaged when necessary and led when appropriate.
4) Unicorn Essence – Finding that magic, unquantifiable element.
This theme of needing to resolve the unresolvable was a constant theme for me, as was the constant hunger for the immersive theatre folks for insights on how become profitable without sacrificing their vision. Lots of passion looking for answers where there no easy ones makes for lots of instantly intense conversations.
Challenge #2: Expanding the magic circle
Several of the presentations touched on the importance of thinking broadly about visitors’ experience, and not just on what you think is “the core” of it. And in the case of VR, it was especially refreshing to hear someone talk about the importance of the holistic experience, not just what you see when you put on the headset.
Ethan Stearns, VP of Virtual Reality at Legendary Entertainment, worked on designing Alejandro G. Iñárritu’s Academy Award-winning CARNE y ARENA (Virtually present, Physically invisible). This groundbreaking VR/physical immersive experience puts viewers in the place of migrants trying to cross the Mexico/U.S. Border. Heady stuff. Interestingly, though, Ethan’s talk focused not on what many might consider “the experience” putting on the headset and viewing the film, but on the importance of introductory space and the concluding experience after you finished the VR. He referred to the intro to Carne as acting like an “amuse bouche” (“amuse the mouth”); the delicacy preceding the meal that awakens the palate and gives you a glimpse of the chef’s style. I think its a great analogy. And in the case of Carne, the amuse bouche was a sterile, cold white room that had the vibe of a detention center, without ever signaling what it was trying to be. It was just the place you started before doing the “real thing”; the VR experience. But it was crucial to setting expectations, and getting visitors emotionally in the right frame of mind, according to Ethan.
This emphasis on onboarding and offboarding is one area where immersive design really shines. Ethan’s talk was a great example of how experience designers need to think holistically, and reminded me of Johanna Koljonen’s description of the user experience.
“Immersive Theatre feels more like a memory than an entertainment.”
Ethan Stearns
Folks to follow #2: Johanna Koljonen
Johanna Koljonen, CEO of Participation Design Agency, is someone whose work I admire deeply and have written about before. Her talk on Nordic larp design found an appreciative audience at IDS, because she has managed to provide a descriptive framework for what a lot of immersive designers have been doing. This was another recurrent theme for me throughout the event: immersive design is at that awkward stage where people who consider themselves one thing or another are finding out that they might all be the same thing, and they’re struggling to speak in terms that are mutually understandable.
Agency and alibis
Johanna’s talk resonated a lot with Ida’s in that she spoke at length (and incredibly quickly) about the importance of interaction alibis; the thing that makes it possible for people (especially adults) to behave in ways that might be considered transgressive or embarrassing normally. The alibi is the invitation to interact, and the signal that you’ll be safe doing so, because players who don’t feel safe can’t be brave. The alibi is the crucial ingredient to get people to act. For her “agency” is insufficient because it’s only the theoretical possibility that you can interact, people still need the alibi to have permission to play.
“If your participants aren’t doing what you want them to do, it’s not your participants who are wrong.”
Johanna Koljonen
Aside: The weirdness of the new experience economy
If you were wondering how much the experience economy has changed everything, real estate developer Winston Fisher showed off plans for Area 15, a new retail development in Las Vegas that is being built as a destination, anchored by Meow Wolf, an artist collective/juggernaut that even didn’t exist fifteen years ago. The fact that a real estate developer talking about experience design could be interesting was in and of itself interesting, and a reminder of just how broad a swath “immersive” cuts through society and how quickly it has insinuated itself everywhere. Sara Thacher put it well on Twitter:
Folks to follow #3: Sean Stewart
Sean Stewart of Magic Leap closed out the first day with a talk on storytelling and mixed reality. I may remain deeply skeptical of Magic Leap until it ships a product, but Sean knows a thing or two about storytelling. In addition writing sci fi novels, he worked on two of the earliest Alternate Reality Games (ARGs); “The Beast”, and “I Love Bees” in the early 2000s. Along with Ken Eklund’s “World Without Oil”, “I Love Bees” was my introduction to ARGs and their potential. His work at Magic Leap continues that exploration of storytelling mixed with audience involvement.
Sean’s opening anecdote involved his daughter coming out of her bedroom crying on the morning of her eleventh birthday because she hadn’t gotten an acceptance letter from Hogwarts. She knew Harry Potter was fiction, but the emotion was all-too-real. A thread throughout his work has been to blend or overlay the real world with storyworlds. All of his non-literary work has resonant echoes of Fri’s “bold blurs”.
Sean compared the way other media have been transformed by new technologies to speculate on how mixed reality (MR) was going to change theatre. Home video streaming has completely shifted the video consumption landscape and affected television and motion pictures irreparably. What if the same thing happened to live theatre? MR and AI make that imaginable now, and MagicLeap is working on just that. He painted a picture of how “home theatre” might work, where your home was the theatre and your MR platform used your physical environment as the setting for entertainment. In this vision of the future, his daughter (wearing a Magic Leap headset) could watch a virtual owl fly through a window in her bedroom and drop a virtual Hogwarts acceptance letter, that would fall on her actual bed and lay there, waiting to be picked up. He laid out three capabilities MR would need to have in order to achieve that vision, all of which ML is working on: 1) Spatial awareness, 2) Someone to play with, 3) Unique ways to play. The rest of the presentation featured a lot of goats; tiny virtual baby goats successfully navigating a real world living room that ML was mapping on the fly, so the kids could do their goaty things like jump onto the couch, or walk under a chair. Impressive stuff.
And then there was Mica, their AI agent that has crossed the uncanny valley into someplace new when it comes to being eerily human.
Challenge #3: Acknowledging the changing relationship between art and audiences
Sean’s talk was full of moments of real geek wonder and technical wizardry but one of his side points has stayed with me. He put up a version of the above slide about the changing relationship between audiences and art as part of his presentation on how home theatre will change to be more like theatre. And he argued something that museums are living through right now; the seismic change in the relationship between people and art. He believes immersive design is at the point of having what he called “it’s Magna Carta moment” about control of content. Until now, the creators has always pulled all the levers, and the audience has consumed. That dynamic has inalterably shifted towards audiences, and he thinks there’s no going back. Time to admit it, and find ways to work with the new reality, just as King John had to accept the barons.
I think this shift applies much more broadly than to just immersive design. I see it all across the landscape. What does it mean for us in the cultural sector? How do we respond to the barons clamoring for rights and privileges we’ve kept to ourselves until now?
Challenge #4: Moving from engagement to involvement
Whereas Day One had a lot of marquee names mentioned (MeowWolf, Punchdrunk, MagicLeap, SXSW) and huge budget projects showcased, Day Two of the summit had a strong social impact theme. The challenge for a whole subgenre of immersive art is, as artist Muna Ahmed put it, “How do we make art work that goes beyond presenting a problem?” How can experience designers leverage the powerful affect of immersive experiences to to not just engage audiences, but involve them in making a more just and equitable world? The morning’s speakers had amazing resumes of creating powerful work: Intelligent Mischief’s “Black Body Survival Guide”, Peoplemovr’s work on “The Mile-Long Opera”
“If you’re not at the table, you’re on the menu.”
– Geoffrey Jackson Scott
Mikhael Tara Garver provided a real challenge to audience about cost and inclusion, reminding us that a ticket isn’t an invitation. In her work, she is always thinking how to manage that by putting the point of charging money in a different place. One of my favorite quotes of the day was, “How do you put the money at the point of most impact and keep it away from where you want the most people to come?” Now there’s a challenge that’s more complicated than “What’s the right ticket price?”
Folks to follow #4: Jenny Weinbloom
Jenny Weinbloom from MeowWolf (and formerly of Punchdrunk) was brought in to close out Day Two and she killed it with a high energy collection of provocations about creators and audiences and what makes a “good” audience. Using examples of her own rule breaking as an audience member, she pushed the audience to think about the inherently aggressive nature of immersive arts and what opportunities that provides creators to build what she called rehearsals for resistance. She asked, “Does the work that we’re creating have to have the same rules as the world that we’re living in?” Or could it strive to present alternate, better versions?
“Immersive work, at it’s best, is an opportunity for resistance. Immersive art is inherently anti-authoritarian.”
Jenny Weinbloom
She also made the point about the distinction between ritual and story. Both are narrative structures, but they operate very differently and have different goals. She talked about how audiences to Sleep No More were always moving props around to create little shrines. Transgressive (certainly to the set dressers) but indicative of some kind of emotion connection being created. But what message were they sending?
Her keynote tied the whole event together for me. Ida’s talk of rituals and spiritual trespass. Ethan’s comment about immersive feeling more like memories than entertainment, Johanna’s point about needing to feel safe to be brave, and Sean’s talk about user control and agency.
Conclusion: groping toward community
Part of the whole impetus for the event has been to build community between a group of people whose day jobs seem pretty disparate; actors, dramaturges, VR developers, writers, theme park developers, etc… Part of the appeal of this event for me was observing the process of people deciding they might be a community, and the joy of learning about different ways of designing “immersive” experiences. The game theorists at the summit had their vocabulary, which was different than the larpers, which was very different from the theatre peoples’. Seeing them all moving towards a common identity and shared vision of what their field might be was deeply enriching. You don’t often get to watch something being born.
Perhaps the biggest indicator of the current state of affairs was the field’s first industry report, put together by No Proscenium. By their estimate, immersive experiences in North America are already a $4.5 billion industry, and that’s without counting theme parks, which generate $45.2 billion yearly. Museums are most notable through their complete absence. Hopefully, we’ll change that next year, right? Get busy!
Check out the IDS report, INTERACTIVE, INTIMATE, EXPERIENTIAL: The Impact of Immersive Design if you want to see what the contours of an emerging industry look like. If nothing else, it’s a great mini White Pages for potential collaborators.
So, there’s a very idiosyncratic view of some of what happened at IDS. If you’re at all interested in doing immersive work, the summit next year might be worth your time.
Additional takeaways
- Being an outlier can be a welcome reset of your assumptions and identity. The vast majority of folks there were doing live action immersive events, followed by AR/VR/ gaming types. I could count the museum people on one hand. It was great to introduce myself to people and have them pause and then nod their heads as they figured out whether museums could be immersive spaces.
- Food matters.
- Events full of experience designers tend to have better slide decks. I didn’t encounter any examples of people just reading their slides to us. People seem to actually rehearse their talks beforehand!
- Hard seats suck.
- Being cold and stationary sucks.
- Events full of actors can be INTENSE. I don’t think I’ve ever been asked to show my enthusiasm so many times in a professional context as I was at IDS. It seemed like every speaker wanted to know how I was doing and If I was having a good time. Made my inner cranky Yankee suspicious, “Why do you want to know?” It was California, though, so I went with it.
- Hollywood producers are actually stone cold monsters. ‘Nuf said.
- Publicly staying on time makes it clear to the audience that you care. Another thing experience designers pay attention to. It’s not enough to care. When you actively demonstrate that care, you lower stress levels in the audience. And there’s something really humanizing to see some luminary get that glint of panic in their eyes when the moderator holds up the “2 MIN” sign and they know he means it.
Thinking about Museums 