MetaHumans are Epic's photoreal characters for Unreal Engine. Their faces are driven by a rig of a few hundred controls, far richer than the 52 ARKit blend shapes. Yet you can animate a MetaHuman's mouth with ARKit curves, because Epic built a bridge for the iPhone.
How ARKit reaches a MetaHuman face
The path was made for Live Link Face, Epic's iPhone app, which streams the phone's 52 ARKit values to Unreal in real time:
- A Live Link source receives the curves by their ARKit names (
jawOpen,mouthFunnel, ...). - The MetaHuman's face Animation Blueprint reads them with a Live Link Pose node.
- A pose asset included with every MetaHuman (
mh_arkit_mapping_pose) turns each ARKit curve into the matching combination of MetaHuman face controls.
So the MetaHuman doesn't need ARKit shapes of its own: anything that produces ARKit-named curves can drive it through that mapping. Asset names and blueprint layouts change between Unreal and MetaHuman versions, so check the current Epic documentation for your version.
Driving the mouth from speech, without an iPhone
An iPhone captures a performer. For an AI character, there is no performer: the curves have to come from the speech itself. Three ways to feed them in:
- As an animation asset. Turn the curves into an Animation Sequence with one float curve per ARKit name, and play it in the face Animation Blueprint before the mapping pose. This is the simplest option for pre-recorded lines.
- As a Live Link source. For live conversation, push the curves for each sentence into Live Link as it is spoken, the same way the iPhone app does. The existing face blueprint then works unchanged.
- With a vendor plugin. Several conversational AI platforms ship their own MetaHuman integration, tied to their pipeline.
Whichever path you use, keep the curves in step with the audio player's clock, and ease into and out of each line so the mouth doesn't snap. The same rules apply as in our three.js lip sync guide.
What the mapping can't do
The ARKit set has no separate tongue tip, no lip-corner tightening and only one tongue shape (tongueOut). Through the mapping, a MetaHuman will speak well, but the extra detail its rig supports, like tongue placement on "l" and "t", stays unused. For close-up hero shots, studios add hand animation on top or use MetaHuman Animator, Epic's tool for solving full facial performance from video.
For an AI host on a lobby screen, ARKit-level lip sync is more than enough: viewers read lip closures, rounding and jaw timing, not tongue placement.
Where MetaHumans fit for a business
A MetaHuman is rendered by Unreal Engine on a GPU, so on the web it is streamed as video (Pixel Streaming), and every live viewer occupies part of a GPU. That suits flagship screens: a hotel lobby, a showroom, an event stand. For every website visitor, a lighter 3D character drawn by the visitor's own device is the better fit. We compare the options in 3D AI avatars vs video avatars.
MetaHuman output from the Lobbyface Lip sync API
Our Lip sync API returns ARKit curves from text plus audio or phoneme timings. A MetaHuman output that follows the mapping above is on the roadmap for early access, so the curves can go straight into an Animation Sequence or a Live Link source.