How to use LipSync for multiple characters

This section describes how to use LipSync for multiple characters.

Note:

Up to 8 CriLipsAtomAnalyzer handles can be generated via the default settings of the plugin. ( CriWare.CriLipsAtomPlugin.InitializeLibrary )
This is because the work size of CriLipsAtomAnalyzer is large and each handle consumes about 140KiB of memory.
Also, when multiple audio streams are analyzed simultaneously, a high CPU load occurs. So, we assume a general case, such as a conversation scene of an adventure game, and set the maximum number to 8.
We recommend using as few handles as possible.
If you still need to create more handles, refer to the section below on how to remove the restriction.

There are two main methods to perform lip sync for multiple characters using the ADX LipSync Library.

When using CriWare.CriLipsDeformerForAtomSource component

Under the following conditions, you can use the CriLipsDeformerForAtomSource component to perform lip sync easily.

• Using CriAtomSource
  
• Up to 8 characters who perform lip-sync can't be swapped throughout the scene
  

First, prepare a CriLipsDeformerForAtomSource component for each character.
In the scene, by separately using the CriAtomSource set above according to each character to enable lip sync of multiple characters.
Refer to the following pages for how to set the CriLipsDeformerForAtomSource component.

• CriLipsDeformerForAtomSource
  

When using CriLipsAtomAnalyzer class from script

This is useful if you are using CriAtomExPlayer and multiple characters appear one after another during the game.

Although there are restrictions mentioned before, you can perform lip sync for multiple characters by using the analyzer as follows.
First, create two CriLipsAtomAnalyzer in advance.

CriAtomExPlayer player1 = new CriAtomExPlayer();
CriAtomExPlayer player2 = new CriAtomExPlayer();
CriAtomExPlayer player3 = new CriAtomExPlayer();
CriLipsAtomAnalyzer atomAnalyzerA = new CriLipsAtomAnalyzer();
CriLipsAtomAnalyzer atomAnalyzerB = new CriLipsAtomAnalyzer();

Next, when performing lip sync in the scene, play the audio after attaching the generated CriLipsAtomAnalyzer to the CriAtomExPlayer to be controlled.
At this time, the mouth shape information analyzed can be acquired from the attached CriLipsAtomAnalyzer.

/* Attach an analyzer to player1 and analyze the audio of player1 */
atomAnalyzerA.AttachToAtomExPlayer(player1);
/* Playback by player1 after attaching */
player1.Start();

/* Attach an analyzer to player2 and analyze the audio of player2 */
atomAnalyzerB.AttachToAtomExPlayer(player2);
/* Playback by player2 after attaching */
player2.Start();

Each time the controlled CriAtomExPlayer changes, detach and attach operations are performed.
Detach can be applied when CriAtomExPlayer.Status is CriAtomExPlayer.Status.Stop.

if(player2.GetStatus() == CriAtomExPlayer.Status.Stop){
  /* Remove and attach the analyzer again to switch from player2 to player3 */
  /* It is also possible to forcibly stop playback by passing true in the argument of CriLipsAtomAnalyzer.DetachFromAtomExPlayer() */
  atomAnalyzerB.DetachFromAtomExPlayer();
  atomAnalyzerB.AttachToAtomExPlayer(player3);
}

Attention

When you need to analyze the voices of 9 or more people at the same time
If you can tolerate the work size of CriLipsAtomAnalyzer mentioned in the opening note and the load of multiple simultaneous voice analysis, you can change the upper limit from the script.
Any number of CriLipsAtomAnalyzer instances can be created by increasing maxHandles of CriLipsAtomPlugin.InitializeLibrary(uint maxHandles) at library initialization.
However, it is recommended to check the operation on the target device because the memory consumption and CPU load will increase.