[CriAtom] Using the LipSync plugin to create lip-sync via script

Sample description

Overview

By creating the component while the application is running,
you can perform real-time lip-sync for the character model.

Operating instructions

• [SetUp Component] button
  
  • Set up the CriWare.CriLipsDeformerForAtomSource component for lip-sync.
    
• [Unsetup Component] button
  
  • Deactivate the CriWare.CriLipsDeformerForAtomSource component for lip-sync.
    
    
• [Playback Cue] button
  
  • Play the audio from CriAtomSource.
    
    

Scene information

Middleware	ADX LipSync(CRI Lips), CRI ADX (CRI Atom)
Sample	Expansion sample
Location	/CRIWARE/SDK/unity/samples/UnityProject/Assets/Scenes/criatom/expansion/LipSync
Scene file	Scene_02_LipSync_ScriptAddComponent.unity
ACF file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acf
ACB file	/CRIWARE/SDK/unity/samples/UnityProject/Assets/StreamingAssets/ADXLipSync/ForADXLipSyncSample.acb

Description of the Program

First, use AddComponent() to add the CriLipsDeformerForAtomSource component in the Start() function
and initialize the LipSync library and create the mouth shape analyzer.

private void Start()
{
    this.criLipsDeformer =  this.gameObject.AddComponent<CriLipsDeformerForAtomSource>();
}

Create an CriWare.CriLipsMorphBlendShapeWidthHeight object based on the data for each morph target of the character model,
and register to CriWare.CriLipsDeformerForAtomSource::LipsMorph .
At this point, the name of the morph target must be converted to an index by calling GetBlendShapeIndex()
and then passed to CriWare.CriLipsMorphBlendShapeWidthHeight::lipHeightIndex .

CriLipsMorphBlendShapeWidthHeight lipsMorph = new CriLipsMorphBlendShapeWidthHeight();

lipsMorph.Target = skinnedMeshRenderer;

// Set morph target index to CriLipsMorphBlendShapeWidthHeight.
lipsMorph.lipHeightIndex
    = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex("blendShape.LipHeightOpen");
lipsMorph.lipWidthOpenIndex
    = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex("blendShape.LipWidthOpen");
lipsMorph.lipWidthCloseIndex
    = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex("blendShape.LipWidthClose");
lipsMorph.tongueUpIndex
    = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex("blendShape.Tongue_Up");

criLipsDeformer.LipsMorph = lipsMorph;

Then by attaching CriWare.CriLipsDeformerForAtomSource to an CriWare.CriAtomSource ,
the audio being played is analyzed by ADX LipSync, and the morph target of the character model is updated accordingly.
Please make sure to select the sampling rate of the audio to be played by the CriAtomSource in CriWare.CriLipsDeformerForAtomSource::atomAnalyzer .

criLipsDeformer.AttachToAtomSource(atomSource)

// Set the sampling rate of the cue to be played.
CriAtomEx.WaveformInfo waveformInfo;
CriAtom.GetAcb(atomSource.cueSheet).GetWaveFormInfo(atomSource.cueName, out waveformInfo);
criLipsDeformer.atomAnalyzer.SetSamplingRate(waveformInfo.samplingRate);