Experiential specialists at The Mill use Unity to create live VR events for media influencers.
The Mill: A Unity case study
How does a creative team deliver captivating live experiences while limited to a high-pressure schedule and restrictive event conditions? The Mill, a global creative production studio, did just that when they came across an exciting opportunity: create a series of social VR events to celebrate HBO’s Lovecraft Country season finale.
| The challenge | Platforms | Project staff | Location |
|---|---|---|---|
| Create exciting immersive experiences despite platform limitations | Oculus Quest, VRChat | Developers, technical artists, concept artists, CG artists, experience designers, writers | London, Los Angeles, New York |
Reaching an isolated audience
Media events in the entertainment industry used to be splashy, multisensory presentations held in ballrooms and theaters – a coordinated way to get mass-media coverage, all at once. But the 2020 coronavirus pandemic changed everything. No more social gatherings, and no more going on-site to physical locations.
So instead, HBO’s series celebration went virtual, inviting press and online influencers to Lovecraft Country: Sanctum. This immersive experience would feature three live events where VR attendees experienced specially created Lovecraft content, all while interacting with their peers as 3D avatars in the social VR platform VRChat.
The Mill crafted a vision for the three events and took on the project with confidence – they had an abundance of creativity; a team with extensive experience with Unity, their primary XR development platform; and a plan to get maximum press coverage from journalists stuck at home.
The results
- Over 100 press features and 13 million social impressions from the first event
- Greater creative freedom with more iterations through real-time production reviews
- Faster production through enhanced VR collaboration
- Efficient remote and distributed workflows across 8 time zones
- Engaging interactive VR environments using < 50 MB per world
Pioneers of virtual production
The Mill was the first fully digital VFX company in Europe when it started up in 1990. Since then, it has created considerable award-winning XR, experiential, and interactive content for the top marketing and ad agencies, production studios, and entertainment networks in the world. According to Sally Reynolds, The Mill’s Design Director for this project, “Our teams aren’t just artists or engineers, they’re creative technologists, and they have a natural curiosity about how to use new techniques to tell stories.”
The Mill has proved the ascendance of virtual production and they use real-time 3D animation and XR technologies for previs, scouting locations, and monitoring shoots as well as virtual sets and characters. “XR lets us open up new workflows that speed up production with unprecedented creativity,” says Technical Director Kevin Young. This level of expertise is one of the reasons why The Mill creates so much of its content in Unity. “We want to be able to completely customize everything, build our own assets and write custom code. Unity lets us do this.”
Selecting a social platform
The HBO media event had to make a big splash in the midst of the pandemic’s chilling isolation. Immersive VR would make for a rich audiovisual experience, but if you’re going to party, you need people to talk to. And how do you add people to a VR session? Interactive chat worlds, of course.
The parties would include 100 top social influencers, who typically are encouraged to dress up and sparkle for media events. That meant robust, customizable VR avatars were a must. With this in mind, The Mill chose to create the Lovecraft Country: Sanctum worlds within VRChat, a free-to-play, massively multiplayer VR social platform that is built in Unity. VRChat is an established and collaborative platform, and it’s very open programmatically with friendly, accessible SDKs. Young says, “The fact that we know Unity meant we’d be able to produce content in VRChat pretty quickly. Anything creative was on the table.”
Creative constraints
The challenges were steep. The Mill had two months, from kickoff to delivery, for a remote, distributed workforce to collaborate and put together cutting-edge XR content. The deadline – tied to the airing date of the series finale – was set in stone. What’s more, content for a VRChat world was limited to 50 MB. With a comparatively low-res VR headset, the worlds inside Sanctum had to somehow do justice to the series, a lavishly produced, high-production value 4K TV show.
Neither HBO or The Mill could take for granted that the audience would be at all familiar with virtual reality. In fact, the team had to assume their audience was filled with total VR newbies.
Tricks and trade-offs
Preproduction kicked off with a massive creative brief that included all the top-level ideas that represented everything that everybody wanted – the sky was the limit, says Reynolds, the team’s design director. Then the dev team picked it apart to figure out what they could try to do within the time constraints. As things progressed, typical feature-scope pressures increased. As artists and designers came up with new ideas, the developers continually tried to make them work with the impossibly small 50MB world sizes.
Interestingly, two standout Unity advantages that enabled the development of Lovecraft Country: Sanctum were also responsible for helping the teams resolve potential conflicts and get the best technical performance for the event. The first was collaboration. “Real collaboration takes personal communication, and this is what an interactive VR world offers. So we held daily meetings in the actual VRChat worlds,” says Young. The other factor was Unity’s superior multiplatform support. “Unity just did extraordinary things within the 50 MB size limit for VRChat and the Quest headset.”
Fantasy over fidelity
Still, the team had to make trade-offs working within that file-size limit. Important hero objects such as the Lovecraft gallery art – created by Black afrofuturist artists David Alabo, Devan Shimoyama, and Adeyemi Adegbesan – required greater detail that couldn’t be artistically compromised. Props in the rest of the world had to be kept simpler to avoid flicker on a headset screen. And the developers tried to minimize any comparison between a 4K linear film series and a social VR experience on mobile VR.
The designers ultimately created haunting new worlds inspired by the otherworldly settings in the story, where the characters raced, fought and explored through space, time, history and fantasy. Each of the experiences reflected this feeling of exploration and focused on visitors’ interactions, allowing them to sense their surroundings, the activities, and each other, rather than attempting to replicate the elaborate series visuals. As Reynolds says, “Occasionally we resorted to old-school visual tricks, but they were never noticeable to the audience.”
Developing real-time collaboration
For building each world, the first step was using Unity ProBuilder to block out environments with massive blocks, Young says, as if it were a game level. This was a quick process and it only used around 1 MB. ProBuilder put a default one-meter (39.4 inch) square grid over the top of the plane/block layout as a scaling guide for artists.
As a level filled in, teams could compile almost instantly to see how it looked within the actual VR environment. “It was so valuable collaborating in Unity in real-time,” says Young. “People would walk around, check it out, and in no time we’d have another iteration.” It worked so well, he says, that the team would actually have client meetings inside the worlds where they could discuss the upcoming day’s work with the stakeholders from HBO.
Unity Nested Prefabs, created from GameObjects inside the project, allowed multiple people on the development team to work inside the scene separately, further enhancing team collaboration and compressing the overall timeline.
Additionally, the team relied on Bakery – a GPU lightmapper with a real-time preview available on the Unity Asset Store. “That was a massive benefit because it meant you didn't have to wait for the entire bake to happen for you to realize what the scene was going to look like,” Young says.
Press party
After receiving their Oculus Quest headsets and donning them at event time, influencers entered an antechamber world to pick an avatar and, if needed, take a quick tutorial to learn VRChat basics. Next, they moved to the Sanctum world to experience the event live. After each show, several mystery worlds unlocked where people could play with various challenges such as an escape-room experience.
The first episode took visitors on a stroll through a whimsical garden with afrofuturist art installations. A live poetry reading inspired by James Baldwin and performed by Lovecraft Country star Jurnee Smollett highlighted the second episode. The final Sanctum event celebrated the finale of Lovecraft Country’s first season with a “Music of the Cosmos” live performance by actress and singer Janelle Monáe.
Did this new approach to media events work? In short, the results – like the worlds – were stratospheric. The Mill’s efforts in social VR garnered over 100 press features and 13 million social impressions in the first week alone. The showrunner herself was reportedly pleased as well. Young says, “In the middle of a pandemic, we brought people together from all over the world. And we had a fantastic response. VR made it happen, and at a fraction of the cost of a live event.”
“Roses are Falling” / Devan Shimoyama
Expanding the possibilities
The success of the Lovecraft Country: Sanctum experience clearly showed the potential of real-time development of social VR for a wide range of applications, from product launches and entertainment premieres to education and cultural events. “We created Sanctum as a space to expand the possibilities of what the future of theatre, games, art installations, concerts, and live events can be,” says Reynolds. And they’ve proved that creative agencies can smash through limits to deliver that future today.
