Facial mouth shape suitable for ADX LipSync

In this section, we will describe the facial composition suitable for the mouth pattern to be output
in order to move the character attractively using ADX LipSync.

The facial mouth shape is prepared using a blend shape.
Mouth shape control using ADX LipSync can be controlled by a combination of vertical and horizontal shapes and a combination of five Japanese vowels.
Since the movement of the tongue can also be acquired, the fidelity of the utterance increases if it is prepared separately.

The following describes the mouth shape for each blend shape based on the model
used in the SDK sample " [CriAtom] Using the LipSync plugin ".

Vertical and Horizontal shape combination

The shape of the mouth is expressed by adding the vertical and horizontal opening of the mouth.
A problem will occur if both the vertical and horizontal blend shapes are maximized at the same time.
Essentially, both vertical and horizontal are rarely maximized,
so it is better to make the bankruptcy limit more conservative for normal mouth movements.
In addition, since the width of the "U" pronunciation is smaller than the mouth shape in the no-sound state,
it can be expressed well by preparing and controlling the "open sideways" blend shape and the "close sideways" blend shape
separately.

In the sample model, three types of blend shapes are available: "Vertical opening", "Horizontal opening" and "Horizontal closing".

Vertical opening (0.0f - 1.0f)

Horizontal opening (0.0f - 1.0f)

Horizontal closing (0.0f - 1.0f)

Japanese 5 vowel combination

Mouth shapes corresponding to the Japanese vowels "A", "I", "U", "E" and "O" are prepared and expressed.
These are a set of general mouth shapes for shaping words.
Up to two vowel blend shapes are output simultaneously, and the total is less than 1.0f.
Since there is no way to move the maximum amount of blends simultaneously, this combination is easy to handle with few failures.

The mouth shape corresponding to each vowel in the sample model is as follows:

A

blend shape (0.0f - 1.0f)

I

blend shape (0.0f - 1.0f)

U

blend shape (0.0f - 1.0f)

E

blend shape (0.0f - 1.0f)

O

blend shape (0.0f - 1.0f)

Tongue movement

From the normal position of the tongue, prepare a shape that is lifted up and stuck to the gums.
In this way, the movement of the tongue in the mouth can be reproduced.

The movement of the tongue by the blend shape in the sample model is as follows.

Tongue position blend shape (0.0f - 1.0f)

Next: Real-time mouth motion control using plug-in components