Tools Overview

This section provides a short overview of the new tools introduced by the Ncam AR Suite plugins for Unreal Engine

An Ncam Data Source is a UE4 asset that handles the connection between the Unreal Engine and an Ncam system (e.g. Ncam Reality, Ncam Emulator). The Ncam Data Source receives information on camera tracking, lens encoding, film image, distortion map, depth map, key image, and key parameters. This asset is a main requirement for all other Ncam assets that need to process information coming from an Ncam server.

Among the settings, an Ncam Data Source specifies an IP address used to reach the Ncam system, and the list of requested packets used inside Unreal.

Ncam Profile is a central control panel that holds a configuration of settings which include the desired input and output sources, frame rate, resolution, time code settings and the assets for synchronizing everything together. It simplifies the setup of any virtual production project and allows for (re)configuration within a few seconds by automating most of the many manual and cumbersome steps.

The Ncam Profile is designed to provide one single place of interaction, controlling all other (time-synchronized) objects, such as Media Sources, Media Proxies, Media Outputs, Timecode Synchronizer, Global Settings (Frame Rate, Standard, Timecode Format etc.), Ncam Data Source, Genlock, and Timecode Provider. The Ncam Profile is built upon Epic Game’s Media Profile[1] and extends it further with a large number of additional features and automated internal processes.

To simplify the synchronization process and the configuration of Live-Link and video sources, Ncam offers a wizard called “Ncam Profile Builder”, which is designed to create Ncam Profiles complying with the user’s needs. This wizard is located in the “Window → Ncam Profile Builder” menu entry of Unreal when the Ncam AR Suite Lite plugin (or the Ncam AR Suite) is enabled.

Furthermore, the Ncam Profile Builder can automatically create an Ncam Data Source, which will be associated with the Ncam Profile the builder is going to generate.

An Ncam Media Source extends the concept of the Unreal Media Source to work with Ncam video streams. Each Ncam Media Source is associated with an Ncam Data Source, which receives frames from an Ncam server (e.g. Ncam Reality, Ncam Emulator). It processes the video frame data coming from the Ncam Data Source, and contains information on video standard, video caching, and type of texture (e.g. film image, distortion map, depth map, key image). To stream the Ncam video frames on a texture, the user can use a Media Bundle and set its source to an Ncam Media Source. The Media Bundle can also be used inside a Timecode Synchronizer, if synchronization is required.

The Ncam Live Link Source exposes Ncam tracking data to Live-Link which allows it to be used to control actors inside the Unreal Engine. For more information on Live-Link, please see the official documentation provided by Epic Games

The Ncam Game Mode is based on the native concepts of Game Modes inside Unreal Engine and simplifies the use of Ncam with Unreal Engine even further. For more information, check the “Ncam Game Mode” section of the Documentation.

The Ncam World Origin allows the datum point (origin) of the Ncam tracking system to be moved, rotated and scaled within the scene. As the virtual camera moves relative to this datum point using an Ncam World Origin allows the virtual camera to be easily repositioned within the scene. The Ncam World Origin is a Blueprint that is located in the Unreal Content Browser under “NcamARSuiteLite Content/BluePrints”. Simply placing it into a scene will adjust and reposition all tracking data relative and the cameras it drives to its own 3D Transformation. This tool is very useful in production as the datum point of the Ncam system rarely corresponds to the origin within the Unreal Scene and often requires rotating or moving to better locate the camera.

For the curious, the tracking data received by the Ncam Live-Link subject is offset using an Ncam Frame Pre-Processor that applies a 3D transform to each frame of incoming data. This transform is updated directly by the Ncam World Origin Blueprint if it is placed into the scene.

Ncam AR Suite Lite provides two example blueprints to illustrate how Ncam tracking data can be used to render and composite CG elements with a video feed. Ncam AR Suite goes further to supplement these with many more rendering features and visual effects. These two example blueprints show how the two most common virtual production use cases can be easily achieved:

• VirtualOverlayExampleCamera: Renders and composites CG elements on top of a video feed;
• VirtualStudioExampleCamera: Renders and composites a video feed over a CG background using an alpha mask. The alpha mask can be produced using either Ncam Reality’s keyer or by an external keyer.

These actors are located inside the “NcamARSuiteLite / Content / Rendering” folder: