Importing into Unreal Engine

This section explains how to import the output results of CriLipsMake2's Mpeg4-Visemes animation curve file (.csv) into Unreal Engine.

Import as DataTable Asset

This is the method used in the sample project included in the SDK. Since the CSV is handled directly, it is easy to replace analyzed data.

The values described in the CSV are directly assigned to the character model.

First, define the data table as the LipsMP4Viseme type in Source/RealLipsync/Public/SmpUtils.h.
This time, only the phoneme columns are targeted for import.

USTRUCT(BlueprintType, Category = "Sample")
struct REALLIPSYNC_API FLipsMP4Viseme
    : public FTableRowBase
{
    GENERATED_BODY()

public:

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float Sil = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float PP = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float FF = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float TH = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float DD = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float KK = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float CH = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float SS = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float NN = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float RR = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float AA = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float E = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float IH = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float OH = 0.0f;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Sample")
    float OU = 0.0f;
};

Import the CSV file output by CriLipsMake2 into the project as LipsMP4Viseme.

To use in BluePrint, implement GetDataTableValue() in Source/RealLipsync/Private/SmpUtils.cpp to find the row that matches the playback time.

FLipsMP4Viseme USmpUtils::GetDataTableValue(UDataTable* InDataTable, float InTime)
{
    check(IsValid(InDataTable));

    if (InTime < 0.0f)
    {
        return FLipsMP4Viseme();
    }

    TArray<TPair<float, FName>> TimeNamePairs;
    for (const auto& Name : InDataTable->GetRowNames())
    {
        float Time = FCString::Atof(*Name.ToString());
        TimeNamePairs.Add(TPair<float, FName>(Time, Name));
    }

    TimeNamePairs.Sort([](const TPair<float, FName>& A, const TPair<float, FName>& B) {
        return A.Key < B.Key;
    });

    for (int32 i = 0; i < TimeNamePairs.Num(); ++i)
    {
        if (InTime <= TimeNamePairs[i].Key)
        {
            return *InDataTable->FindRow<FLipsMP4Viseme>(TimeNamePairs[i].Value, FString());
        }
    }

    if (TimeNamePairs.Num() > 0)
    {
        return *InDataTable->FindRow<FLipsMP4Viseme>(TimeNamePairs.Last().Value, FString());
    }

    return FLipsMP4Viseme();
}

In ModelBP, assign the contents of the data table obtained from the playing audio to the character's phoneme morph targets.