Video Playback using Player class

Components such as CriManaMovieMaterial and CriManaMovieController implement CRI Sofdec 's video playback processing that matches the lifecycle of a Unity application.
By using these, you can easily play videos.

It is also possible to independently implement a similar implementation using the Player( CriWare.CriMana.Player ) class without using the components provided by the plugin.
Use it as necessary, such as when controlling the timing of screen updates in detail according to the application specifications.

This chapter explains video playback processing using the Player class.

Implementation of video playback processing

Overall picture of processing

Video playback processing using the Player class is achieved using the following flow.

• Spawn Player
• Set files/data to play
• Start playing
• Regularly update Player state and frames
• Stop playback and destroy Player

Generate the Player

Generate a CriWare.CriMana.Player from a script.
Make sure to maintain an instance in the class that manages playback.

CriMana.Player player;

void SetupPlayer()
{
    player = new CriMana.Player();
}

Set the file/data to be played on the Player

Set the file to be played or the data in memory for the Player.
When setting data in memory, explicitly fix the memory to avoid memory movement due to garbage collection.

// When specifying a file path
void SetFilePath()
{
    player.SetFile(null, System.Path.Combine(Application.temporaryCachePath, "FileName.usm"));
}

System.Runtime.InteropServices.GCHandle gcHandle;

// When specifying data in memory
void SetData()
{
    gcHandle = System.Runtime.InteropServices.GCHandle.Alloc(buffer, System.Runtime.InteropServices.GCHandle.Pinned);
    player.SetData(gcHandle.AddrOfPinnedObject(), buffer.LongLength);
}

Command the Player to play

Instructs the Player to prepare for playback, and starts playback when ready.
When playing a video using CRI Sofdec , there is a time lag from the playback start command (Start function) until it can actually be drawn.
Therefore, we recommend that you call the playback preparation command (PrepareForRendering function), which includes drawing preparation, and then call the Start function and enable drawing at the same time.

IEnumerator StartPlaybackVideo()
{
    // Preparation for playback
    player.PrepareForRendering();

    var status = player.status;

    // wait for readiness
    while (status != CriWare.CriMana.Player.Status.ReadyForRendering)
    {
        if (status == CriWare.CriMana.Player.Status.Error)
        {
            player.Stop();
            yield break;
        }
        yield return null;
        // update status
        UpdatePlayer();
        status = player.status;
    }

    // Start playing and displaying
    player.Start();
    SetMaterialToTargetObject();
}

Update Player state and frame

The Player class performs the following processing regarding video playback.

1. Player status update
2. Texture drawing update
3. Reflecting the drawing on the material

Call the update function accordingly as follows.
The update function must be called periodically, such as within MonoBehaviour.Update, in all states: "preparing to play," "playing," and "finishing playback."

Material material;

void UpdatePlayer()
{
    // 1. Update Player state
    player.Update();
    // 2. Update texture drawing
    player.OnWillRenderObject(null);
    // 3. Reflect drawing to material
    isMaterialUpdated = player.UpdateMaterial(material);
}

The return value of the CriWare.CriMana.Player.UpdateMaterial function indicates whether the appropriate settings and textures have been reflected in the Material passed as an argument.
If this return value is true, there is no problem using the Material for drawing.
If it is false, there is a possibility of incorrect drawing, so please be careful not to display it.

UnityEngine.UI.Graphic target;
Material originalMaterial;

void SetMaterialToTargetObject()
{
    // Should be called when UpdateMaterial is in a successful state
    UnityEngine.Debug.Assert(isMaterialUpdated);

    if (originalMaterial == null)
    {
        // Remember the original Material
        originalMaterial = target.material;
    }
    target.material = material;
}

Stop playback and destroy the Player

Stop playing the video by calling the CriWare.CriMana.Player.Stop function.
There is a time lag between when playback starts and when it stops, so be careful to check the status.
Also, since the stop process that starts with the Stop function also destroys the resources related to drawing, the state of the Material becomes undefined from the moment it is called.
Disable display using Material at the same time as calling the Stop function.

A stopped Player can be reused, but when you are no longer using it, you must explicitly call the CriWare.CriMana.Player.Dispose function to discard it.

void StopAndDisposePlayer()
{
    // Disable the display of Material at the same time as the stop command
    player.Stop();
    ResetMaterialOfTargetObject()

    var status = player.status;

    // Wait for the stop to complete
    while (status != CriWare.CriMana.Player.Status.Stop)
    {
        yield return null;
        // update status
        UpdatePlayer();
        status = player.status;
    }

    //destroy the Player
    player.Dispose();
    player = null;

    if (gcHandle.IsAllocated())
    {
        gcHandle.Free();
    }
}

void ResetMaterialOfTargetObject()
{
    if (originalMaterial != null)
    {
        // Set the original Material
        target.material = originalMaterial;
        originalMaterial = null;
    }
}

Responding to application state changes

Unity applications change state depending on user operations and system behavior.
For stable video playback, it is necessary to respond appropriately to such state changes.

Response to poses

If your application pauses while a video is playing, you should also pause the video playback.
If you do not pause correctly, the time referenced by the CRI Sofdec library will advance, causing problems such as the video speeding up after you unpause the application.

Because some platforms require special handling when suspending and resuming, the Player class provides a dedicated pause function ( CriWare.CriMana.Player.PauseOnApplicationPause ).
Let's call this inside the MonoBehaviour.OnApplicationPause function.

void OnApplicationPause(bool flag)
{
    player.PauseOnApplicationPause(flag);
}

Advanced usage

I want to continue drawing even when playback is stopped

As mentioned earlier, when you call the CriWare.CriMana.Player.Stop function, the drawing content of the Material becomes undefined.
However, as shown below, there are cases where you want to continue displaying the Material even after the video has stopped playing.

• Move the playback position by seeking and then play again
• Switch to another video and play again

In such cases, please stop using the CriWare.CriMana.Player.StopForSeek function.
The Material remains visible (CriWare.CriMana.Player.UpdateMaterial returns true) and further playback processing can begin.

Please note that the file played after calling StopForSeek must have the same codec and resolution as the file before stopping.
If they are different, internal resources will need to be regenerated, which may result in incorrect drawing.

I want to have detailed control over the frame update timing

When playing Sofdec videos, the displayed frames are updated in sync with the system time or the playback time of the audio included in the video file.
Depending on the performance during the game, there are cases where you want to update the frame at a unique timing as shown below.

• I want to advance playback according to the audio playback time outside the video file.
• In order to synchronize with other performances, I want to proceed with playback one frame at a time each time the application frame is updated.

In such cases, set the Player's master timer type and then call the corresponding state update function.

Please note that if you change the master timer type from the default, make sure that the video file does not include audio.
The video frame and audio progression may become out of sync, which may prevent normal playback.

Update by user time

If you want to play the video in synchronization with a unique timer such as the playback time of another module, update the status based on the user time.
Call the CriWare.CriMana.Player.UpdateWithUserTime function instead of the regular Update function.

// Before starting playback: Set to use user time
player.SetMasterTimerType(CriWare.CriMana.Player.TimerType.User);

// Record playback start time
float startTime;

void UpdatePlayer()
{
    // When updating: Update by specifying user time
    var playbackTime = Time.time - startTime;
    var playbackTimeUlong = (ulong)(playbackTime * 1000.0f);
    player.UpdateWithUserTime(playbackTimeUlong, 1000ul);

    player.OnWillRenderObject(null);
    isMaterialUpdated = player.UpdateMaterial(material);
}

Update with manual timer

If you want to proceed with playback at a fixed width every time you update, such as one frame for each application frame, update using a manual timer.
Call the CriWare.CriMana.Player.UpdateWithManualTimeAdvanced function instead of the regular Update function.
Each call to the UpdateWithManualTimeAdvanced function advances the time, so be careful to only call the update function at appropriate times after playback begins.

// Before starting playback: Set to use manual timer
player.SetMasterTimerType(CriWare.CriMana.Player.TimerType.Manual);
// Before starting playback: Set the time update unit for the manual timer
var fps = (ulong)UnityEngine.Application.targetFrameRate;
player.SetManualTimerUnit(1ul, fps);

void UpdatePlayer()
{
    // When updating: Update while advancing the manual timer
    player.UpdateWithManualTimeAdvanced();

    player.OnWillRenderObject(null);
    isMaterialUpdated = player.UpdateMaterial(material);
}

---

   THIS SERVICE MAY CONTAIN TRANSLATIONS POWERED BY GOOGLE. GOOGLE DISCLAIMS ALL WARRANTIES RELATED TO THE TRANSLATIONS, EXPRESS OR IMPLIED, INCLUDING ANY WARRANTIES OF ACCURACY, RELIABILITY, AND ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.