[CriAtom] Reusing CriLipsDeformerForAtomSource using LipSync plugin

Sample description

Overview

Reuse one CriLipsDeformerForAtomSource to lip-sync multiple characters.

Scene information

Middleware	ADX LipSync(CRI Lips), CRI ADX (CRI Atom)
Sample	Expansion sample
Location	/CRIWARE/SDK/unity/samples/UnityProject/Assets/Scenes/criatom/expansion/LipSync
Scene file	Scene_05_LipSync_ReuseCriLipsDeformer.unity
ACF file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acf
ACB file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acb

Description of the program

Reuse one CriLipsDeformerForAtomSource to lip-sync multiple characters.

CriWare.CriLipsAtomAnalyzer , which is managed internally by CriLipsDeformer, consumes a lot of unmanaged memory.
Therefore, to prevent the creation of unnecessary handles, the maximum number of CriLipsAtomAnalyzers that can be created is determined when the Lips library is initialized.

• CriWare.CriLipsAtomPlugin.InitializeLibrary
  

Instead of preparing multiple CriWare.CriLipsAtomAnalyzer , we recommend reusing the generated CriWare.CriLipsAtomAnalyzer .

About reuse

The following resources are prepared for reusing in the sample.

• CriWare.CriAtomSource
  • 2 (per character)
  • Audio playback player that plays the character's voices
• CriWare.ICriLipsMorph
  • 2 (per character)
  • The instance that lip-syncs with the character model
• CriWare.CriLipsDeformerForAtomSource
  • 1 (number of simultaneous speakers in the scene)
  • Lip sync audio analyzer

Each resource is described below.

CriAtomSource

Play Cues for the character's voices.
In the sample, each character has one of the components.

Note:

This sample was created with the assumption that an audio playback player is provided for each character.
Audio playback players do not necessarily need to be separated by character. In that case, please skip the call to CriWare.CriLipsDeformerForAtomSource.AttachToAtomSource mentioned below.

private CriAtomExPlayback PlayAndLipSync(CriLipsDeformerForAtomSource lipsDeformer, CriAtomSource atomSource, ICriLipsMorph lipsMorph) {
    // Do not replace analyzer
    // var result = lipsDeformer.AttachToAtomSource(atomSource);
    Debug.Assert(result);
    lipsDeformer.LipsMorph = lipsMorph;
    UpdateSamplingRate(lipsDeformer.atomAnalyzer, atomSource);
    return atomSource.Play();
}

ICriLipsMorph

This is a class that actually performs lip syncing of character models. Be sure to prepare one for each character.
To generate an instance from a C# script, do the following:

private ICriLipsMorph criLipsMorphForRingo = null;
private void Start() {
    criLipsMorphForRingo = new CriLipsMorphBlendShapeJapaneseVowel()
    {
        Target = ringo,
        aIndex = ringo.sharedMesh.GetBlendShapeIndex("MOUTH_A"),
        iIndex = ringo.sharedMesh.GetBlendShapeIndex("MOUTH_I"),
        uIndex = ringo.sharedMesh.GetBlendShapeIndex("MOUTH_U"),
        eIndex = ringo.sharedMesh.GetBlendShapeIndex("MOUTH_E"),
        oIndex = ringo.sharedMesh.GetBlendShapeIndex("MOUTH_O"),
    };
}

If set to CriWare.CriLipsDeformerForAtomSource.LipsMorph , that class will be used for lip syncing from next time.

private CriAtomExPlayback PlayAndLipSync(CriLipsDeformerForAtomSource lipsDeformer, CriAtomSource atomSource, ICriLipsMorph lipsMorph) {
    lipsDeformer.LipsMorph = lipsMorph;
}

At that moment, CriWare.ICriLipsMorph.Reset of the ICriLipsMorph originally set to CriWare.CriLipsDeformerForAtomSource.LipsMorph will be called and moved to the mouth-closed state.

CriLipsDeformerForAtomSource

There is alip sync audio analyzer internally.
It is responsible for the reference relationship between AtomSource and ICriLipsMorph.

• CriWare.CriLipsDeformerForAtomSource.AttachToAtomSource
• CriWare.CriLipsDeformerForAtomSource.LipsMorph

We recommend placing a fixed number of resources in a scene that requires lip sync, and providing references to each resource when needed, as in this sample.