CRI Sofdec(Unreal Engine) v1
Introduction 08 How to play movies with color space and gamma correction settings

In this chapter, we will show you how to set color space and gamma settings to play Sofdec movies.

Movie playback with color space settings

Color space conversion

The Unreal Engine uses a floating point, linear RGB color space.
This means a wide range of colors for rendering the scene without gamma correction.
Sofdec movies, on the other hand, are encoded into YUV in a different color space than Unreal Engine (usually 8-bit integer in Rec 601 or Rec 709) to allow for better video compression.

Due to differences in formats and color spaces, the CriWare plugin converts the YUV colors used in your movie to the RGB colors used by the Unreal Engine.

Texture from Sofdec Movie: ManaTexture

ManaTexture is a UTexture-derived class that holds sub-textures called "component textures".
Each texture holds a sub-pixel channel from a decoded movie.

A texture class that holds 3 (Y,U,V) or 2 (Y,UV) or 1 (RGB) + alpha channel for alpha movies.

Depending on the platform and movie format (SofdecPrime, H264, or VP9), component textures may be reversed internally, but the color channels used by the UE5 UI and APIs are always 3 (Y, U, V).

When using materials with component textures

Conversion should be done in Blueprints.
The Material Function "Mana Color Space Converter" serves this purpose.

A Blueprint Material always uses three textures (Y,U,V), but the shader compiler may automatically generate a different number of texture samples depending on the movie format and platform.
For example, for H264 on iPhone, the movie decoder outputs to Y and UV textures (NV12 format), resulting in a material using only two textures.

When the "Render to Target" option is checked in ManaTexture

The YUV to RGB conversion is done offscreen and the resulting pixels are placed on the inner surface of the ManaTexture.
Therefore, ManaTexture is an RGBA texture that can be used as a classic texture.

This extra offscreen rendering is not recommended for large movie files (4K, VR...) as it may use more memory and processing.

How to set the color space for movie playback

When using a material with component textures (Y, U, V)

The YUV to RGB conversion is done by the "Mana Color Space Converter" node.

To change the color space, select the "Mana Color Space Converter" node and in the details panel select the "Color Space" you want to use for color transfer.

When using a material with a ManaTexture rendered to an RGBA texture

The color space is actually fixed at Rec.601.

Movie playback with gamma correction

Gamma correction controls the overall brightness and contrast of an image.

As explained in "About playing movies with color spaces set", Sofdec movies and Unreal Engine use different color spaces.
While Unreal Engine uses linear RGB (1.0) without gamma correction, Sofdec movies use non-linear gamma correction (using a factor of 2.2 to 2.4) to convert the YUV values used to sRGB.

The CriWare plugin allows you to control the coefficients used in the conversion by using a function that removes the gamma correction from sRGB to linear RGB.

When using a material with component textures (Y,U,V)

Gamma correction is performed using the "Mana Color Space Converter" node.
There are two ways to change the coefficients used in gamma correction.

Change it from the details panel

Select the "Mana Color Space Converter" node and go to the details panel.
Check "Use Gamma Exponent" and set the desired gamma value in "Const Gamma".
The default is "2.2".
A higher value makes the image darker and a lower value makes the image brighter.

Change it via the node's gamma pin

Connect the variable float output node to the "Gamma" pin of the "Mana Color Space Converter".

The gamma value can be a fixed value or a dynamically changed Material Parameter.

When using a material with a Mana Texture rendered in RGB

Double-click the Mana Texture generated from the imported Sofdec movie.
Change the gamma value in “Gamma Adjustment” under “Render to Texture” in the Mana Texture details.

If the value is set to 0.0, the default sRGB to RGB Linear function will be used.

Setting the value to anything other than 0.0 will cause a gamma adjustment factor to be used.
A higher value makes the image darker and a lower value makes the image brighter.
A value of 2.2 gives a linear function with results equivalent to the default sRGB.

Movie playback with HDR rendering

As explained in About Color Space Movie Playback, the Unreal Engine uses the RGB linear color space with at least 16-bit floating point per channel.
This means that a wide range of colors and brightness is used to render the scene, and this process is called 'HDRR' (High Dynamic Range Rendering).

Sofdec movies are encoded in YUV with up to 8-bit integer per channel.
The range of colours and brightness used is called 'SDR (Standard Dynamic Range)'.

The color space conversion performed by the CriWare plugin is from SDR to HDR.
Some movies may lose dynamic range after conversion and may become inaccurate.
This phenomenon often occurs in scenes with high-intensity lighting.

When using a material with component textures (Y,U,V)

The conversion to linear color space is done within the material to render the desired scene.

When using a material with a Mana Texture rendered in RGB

Conversion to linear color space is done on an 8-bit per channel RGBA surface.
This can result in loss of color and brightness accuracy and the appearance of color banding.

As a workaround, check "Enable HDR" in the "Render to Texture" section of the Mana Texture details panel. This switches the format of the Mana Texture's target surface to 16-bit float per channel, which preserves the range of colors produced by the conversion, but uses twice the amount of memory.

The higher the accuracy, the less visible the color stripes will be.

Unreal Engine Manual Reference

For more information about rendering and materials, see the Unreal Engine documentation.

Rendering & Graphics http://api.unrealengine.com/JPN/Engine/Rendering/
Materials http://api.unrealengine.com/JPN/Engine/Rendering/Materials/