[CriAtom] Using the Native API wrapper of the LipSync plugin

Sample description

Overview

This is a sample that only uses the native API wrapper class (without using the lip-sync component).

Operating instructions

• [Playback Cue] button
  
  • Plays the audio.
    
    
    

Scene information

Middleware	ADX LipSync(CRI Lips), CRI ADX (CRI Atom)
Sample	Expansion sample
Location	/CRIWARE/SDK/unity/samples/UnityProject/Assets/Scenes/criatom/expansion/LipSync
Scene file	Scene_03_LipSync_NativeAPI.unity
ACF file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acf
ACB file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acb

Tips

To prepare several CriLipsAtomAnalyzer and to attach / detach them to several CriAtomExPlayer, refer to:

• How to use LipSync for multiple characters

Description of the program

In this sample, no component is used. Instead, a class that wraps the native API is used.

Library initialization

Before using the native API wrapper class of the LipSync plugin, you need to initialize the library.
CriWare.CriLipsAtomPlugin::InitializeLibrary Initialization of the LipSync library cannot be performed by CRIWARE Library Initializer component or CriWare.CriWareInitializer::Initialize .
When using the LipSync plugin component, the initialization of the LipSync library is done internally by the component.

From creating to attaching the Atom linked mouth shape analysis module

Create the Atom linked mouth shape analysis module CriWare.CriLipsAtomAnalyzer .
If the maximum sampling rate of the audio to be played back by the CriAtomExPlayer exceeds 48000 Hz ,
you need to pass the rate to the constructor of the CriLipsAtomAnalyzer.

atomAnalyzer = new CriLipsAtomAnalyzer();
atomExPlayer = new CriAtomExPlayer();
atomAnalyzer.AttachToAtomExPlayer(atomExPlayer);

Then, attach the player by calling the CriWare.CriLipsAtomAnalyzer::AttachToAtomExPlayer function.

Setting up the Atom linked mouth shape analysis module

Set the parameters according to the environment in which the CriWare.CriLipsAtomAnalyzer object is used.
The parameters of CriLipsAtomAnalyzer can be set even before attaching it to CriAtomExPlayer.
Please refer to each API's reference to learn more about when its parameters can be set.

private CriLipsAtomAnalyzer atomAnalyzer;
~~
atomAnalyzer.SetSilenceThreshold(-40.0f);

CriAtomExAcb acb = CriAtom.GetAcb(cueSheetName);
atomExPlayer.SetCue(acb, cueId);

CriAtomEx.WaveformInfo waveformInfo;
acb.GetWaveFormInfo(cueId, out waveformInfo);
atomAnalyzer.SetSamplingRate(waveformInfo.samplingRate);

Acquiring the mouth shape information from the Atom linked mouth shape analysis module

Get the mouth shape information from the CriWare.CriLipsAtomAnalyzer you created.
To apply the mouth shape information that was returned on the actual character model, it is recommended to use the delta with the closed mouth shape information,
that can be obtained with CriWare.CriLipsMouth::GetInfoAtSilence .

For details about the mouth shape information that can be obtained, please refer to the following page:

• Mouth pattern information that can be acquired with ADX LipSync

private void Update()
{
    atomAnalyzer.GetInfo(out info);
    Vector3 localScale;
    localScale.x = info.lipWidth;
    localScale.y = info.lipHeight;
    localScale.z = info.tonguePosition;
    image.transform.localScale = localScale;
}

Destroying the Atom linked mouth shape analysis module

Destroying the CriWare.CriLipsAtomAnalyzer
Call CriWare.CriLipsAtomAnalyzer::DetachFromAtomExPlayer to detach the analyzer from the CriWare.CriAtomExPlayer before destroying it.

atomAnalyzer.DetachFromAtomExPlayer(true);
atomAnalyzer.Dispose();

Finalizing the library

Call CriWare.CriLipsAtomPlugin::FinalizeLibrary to finalize the LipSync library.