About the mouth patterns that can be obtained

There are two parameter types for the mouth pattern that ADX LipSync Output as an analysis result. It can be used according to the mouth control pattern of the character.

mouth shape information
The vertical opening of the mouth, the width, and the height of the tongue.

/* mouth shape information Structure */
public struct CriLipsMouth.Info {
    public float lipWidth; /* Mouth width (0.0f to 1.0f) */
    public float lipHeight; /* Mouth height (0.0f to 1.0f) */
    public float tonguePosition; /* Tongue position (0.0f to 1.0f) */
    bool isLipWidthReleased; /* Is the mouth width in transition to closed state */
    bool isLipHeightReleased; /* Is the mouth height transitioning to the closed state */
    bool isLipTongueReleased; /* Is the tongue position transitioning to the closed state */
}

It is suitable for use in models that control the mouth with direct parameters such as length and width, or when expressing the mouth shape by scaling a simple figure.
In addition, if you want to express lip sync using only one axis, the opening of the mouth,
you can use only the vertical opening to create a mouth that matches the voice, such as wide opening for "a" but not wide for "i". movement can be realized.
The height of the tongue (vertical position) can also be obtained as mouth shape information.
It is presented as 0.0f when there is no sound, and 1.0f when the tongue is in contact with the gums.
By controlling the tongue as well as the mouth movements, more realistic character expressions are possible.

Attention

The Output of the "tongue height" parameter is a beta feature under development.
The quality of analysis results is not guaranteed. Additionally, change in specifications and behavior may occur in future updates.
Please understand that this is a beta version feature before using it.

Tips

In the silent state, mouth shape information with a height of 0.0f and a width of 0.583f is Output. This number can be obtained with CriLipsMouth.GetInfoAtSilence.
The width increases when you pronounce "i" and decreases when you pronounce "u", based on the value in the silent state.
Therefore, when performing lip sync by directly referencing CriLipsMouth.Info on the application side, please convert the numerical values according to the data structure of the character.

Japanese 5 vowel morph target blend amount
This is the blend amount of the 5 Japanese vowels (aiueo).

/* Japanese 5 vowel morph target blend amount Structure */
public struct CriLipsMouth.MorphTargetBlendAmountAsJapanese {
    public float a; /* Blend amount of morph target "a" (0.0f or more and 1.0f or less) */
    public float i; /* Blend amount of morph target "i" (0.0f or more and 1.0f or less) */
    public float u; /* Blend amount of morph target "u" (0.0f or more and 1.0f or less) */
    public float e; /* Blend amount of morph target "e" (0.0f or more and 1.0f or less) */
    public float o; /* Blend amount of morph target "o" (0.0f or more and 1.0f or less) */
}

This information is suitable for controlling a model with a blend shape of five Japanese vowels. As you speak, the amount of blending of two vowels at the same time increases or decreases within a range of 1.0f or less in total. By mixing the shapes that represent the two vowels, it is possible to express natural mouth shapes even for sounds other than Japanese.

pattern combination mouth shape information and "Japanese 5 vowel morph target blend amount" can be obtained simultaneously in a single analysis. This allows you to use different patterns in combination, such as "aiueo + Tongue height."

THIS SERVICE MAY CONTAIN TRANSLATIONS POWERED BY GOOGLE. GOOGLE DISCLAIMS ALL WARRANTIES RELATED TO THE TRANSLATIONS, EXPRESS OR IMPLIED, INCLUDING ANY WARRANTIES OF ACCURACY, RELIABILITY, AND ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.