Is this the world’s first full virtual-reality motion/voice capture session?

Actors combine mo-cap suit, HTC Vive to become characters, see virtual teleprompters.

Enlarge / Virtual teleprompters? That's pretty rad, Cloudhead Games.

https://youtu.be/9R0TVLGne3U

Forget latex suits and white ping-pong balls. Motion-capture sessions for video games and films have only gotten more intense over the years, thanks to advances like improved, LED-loaded motion-capture outfits and the ability to see robust TV-screen renders of an actor's performance as soon as a take is complete.

Of course, not every studio has a Peter Jackson-caliber budget for motion capture facilities, but the small development team behind upcoming VR game The Gallery: Call of the Starseed found an affordable path to capturing a human actor's performance—and then remembered they already had one cutting-edge gadget handy: the dev kit for the upcoming HTC Vive virtual reality system. The result, shown off in the studio's latest development diary on Tuesday, may very well be the world's first documented use of VR in a motion capture session.

"We wanted the actor to feel as if they were acting on a stage," Cloudhead Games staffer Mike Wilson wrote at the company's blog. As such, after making actor and motion-capture veteran Adrian Hough (Rise of the Planet of the Apes) put on a suit made by mo-cap company Noitom, the designers also asked him to wear a Vive headset. HTC's system enables room-scale tracking, so that users can walk around up to roughly 225 square feet of real space, which means Cloudhead was able to virtually transport Hough into the shoes of his in-game character, the Watcher.

As he walked and moved in real space, he saw himself do so within the world of The Gallery. And he could trigger certain in-game events based on what pace he walked and how he moved, so that his own pace of acting and speaking (captured by a wireless mic) could be more flexible (though the Cloudhead team admits it had to manually trigger some of the content during Hough's performance). What's more, all of the game's contributing actors were able to see hovering, virtual teleprompters within the game's virtual space, complete with all of their dialogue.

“An entirely new game” for the actors

"Essentially, we designed and scripted an entirely new game, just for the actor performance," Wilson wrote, adding that the team had always planned on showing its actors how the game would look in virtual reality. Going to this next level required only a few internal tests to check its feasibility and make sure a headset wouldn't obstruct any motion-capture tracking.

"In normal motion capture, you have to imagine where everything is and find X points to look at, and a lot of it comes from your imagination," Hough said in comments provided to Ars Technica by Cloudhead. "Whereas in VR, you are inside the environment and experience what it is like inside the world playing the character."

In an e-mail interview, studio co-founder Denny Unger told Ars that it was a pretty easy sell to get actors to put on the unreleased HTC Vive headset for the sake of motion-capture acting: "Adrian [Hough] commented [to us] on how this trumped some of the work he had done prior, primarily because it brought the experience of capturing performance to a heightened state of reality not achievable any other way."

When pressed about whether Cloudhead might ever make its VR motion-capture toolset available for other developers, however, Unger was a little more coy, telling Ars, "As with all innovation happening in this space, we are working as hard and as fast as we can to define what this new capability ultimately means for developers and third party interests."