Film and TV production workflows revolve around efficiency and authenticity. Major game development studios follow the same suit, pushing out an abundance of content and competing to create the next best thing. And when it comes to animating digital characters, motion capture is still the best method available.
AI motion capture entered the market – in real terms – in 2022, becoming a potential time-saving solution for developers and animators. It presents a tempting alternative to marker-based tools that include both optical and inertial techniques. There’s no denying that what AI mocap can deliver is impressive, but it’s not quite ready for high-end use.
AI motion capture technology uses machine learning algorithms to extract key points on a person's body from video footage. It uses a simple setup with as few as one camera (which can be a smartphone), and can be completed anywhere, provided you have enough light.
Depending on the software, the data transfer can be instant, or obtained after processing camera footage. But it’s important to know that the ML datasets that train AI tools here are based on optical and inertial motion capture data – the AI model is essentially using a ‘best guess’ interpretation of that data. And while it does a great job of that process, there’s another important aspect to note: it’s not physically accurate just yet.
Markerless motion capture technology like this interprets images to understand motion and joint position. The keyword there is “interprets” – there’s no direct connection to the body, which can sometimes lead to inaccurate or incorrect capture data that has to be edited in post-production before animation work can begin. This isn’t necessarily a problem on smaller productions or sequences where a little massaging of the data isn’t the end of the world, but major productions require the highest degree of accuracy, versatility, and reliability. There would be too much data to wade through and fix on something like Planet of the Apes, for example.
The appeal of AI motion capture is twofold: cost and ease of use. Traditional optical motion capture systems require several – sometimes hundreds – of cameras in order to generate data, which can price productions out of the market. But it’s much more accurate. AI mocap is faster to set up and cheaper to run, but sacrifices a little accuracy to do the work. The alternative is utilizing inertial motion capture suits with embedded IMUs (Inertial Measurement Units), like the Xsens suits. These gyroscopic-enabled markers are strategically placed on the body to measure joint angles. And because the measurements are made on the body, no cameras are needed in the process at all, which means it’s as simple as putting on a suit, calibrating the system, and recording. Setup takes less than 15 minutes – much faster than optical motion capture.
Inertial systems like Xsens have the added advantage of being portable. With no cameras involved in the process, there’s a freedom that comes with this kind of motion capture. Hasraf ‘HaZ’ Dulull recently recorded mocap artist Ace Ruele in a grading suite for his film Max Beyond using exactly this approach.
What’s important is to strike the right compromise for any given production or shot. Optical motion capture is still one of the most accurate ways to generate data, but it requires a full stage and many, many cameras to work. AI systems are much cheaper and easier to setup, but sacrifice data accuracy. And while inertial mocap requires suits, it provides the most accurate data without needing full volumes like optical mocap or camera setups like both optical and AI.
Video by Paul Metcalfe comparing AI Mocap versus Xsens Link and Xsens Animate.
For now, AI motion capture doesn’t quite meet the standards of large-scale production houses, as can be seen in Paul Metcalfe's comparison video. But its integration alongside alternative methods is proving to be an interesting step forward. AI is also being deployed inside the software that analyzes mocap data. That’s not to say the technology will stop developing, but getting accurate motion capture data from this method just can’t happen with the current tools at hand. Considering the ups and downsides, all three technologies have their place in the process and can even complement each other.
However, it’s clear to see that a change in workflow from traditional motion capture is needed in the industry, both to cut down on time and retain precise results. Inertial motion capture is able to fulfill this, and its integration has begun in some major studios. Game developers in particular are quickly catching onto its ease of use and stellar outcomes.
Take a look at Xsens inertial motion capture products to find out more.