Selecting a Workflow

There are a number of ways that Ncam data can be used with the Unreal engine. For example to drive an LED volume using nDisplay or to render a virtual studio by replacing a green screen with a CG background. The Ncam AR Suite plug-in doesn't force a particular workflow on the user and instead aims to simplify and support however they wish to work.

There are multiple ways to composite AR content together with the Unreal Engine but regardless of the workflow there are always a few necessary or common components: media inputs and outputs, camera tracking data, lens data and a way to synchronize it all.

Building and configuring all of these components together can be time consuming, inflexible and error prone. The Ncam Workflow Wizard aims to streamline this process by automating the generation and configuration of these assets for the user. Once created, a method of compositing can be chosen and implemented.

This guide aims to illustrate the various workflow options available to the user, how to choose one and the pros and cons of each.

LiveLink Subjects

The camera tracking data provided by Ncam Reality using it's two SDKs (the standard and lite versions) is made available to the Unreal engine via LiveLink. Each LiveLink Subject provides different data which changes their capabilities and consequently the what workflows they support. This is illustrated in the table below:

The choice of Ncam SDK effects which LiveLink subjects are available. The standard Ncam SDK enables the NcamLens, NcamCamera and NcamMarkers LiveLink subjects. The Ncam Lite SDK only enables the NcamLens LiveLink subject. This and the capabilities of each should be considered when selecting which to use.

Compositing Workflows

Once the supporting assets have been created by the Workflow Wizard a method of compositing the CG with the film needs to be chosen. The two main solutions for this are to either use the Ncam AR Suite rendering pipelines or Unreal's Composure framework.

Composure with LiveLinkLens

Compatible LiveLink Subjects: NcamCamera, NcamLens

The latest version of Composure adds support for lens distortions through the LiveLinkLens plug-in. This allows for a simpler set-up than the legacy composure workflow at the cost of support for anamorphic lenses. Because it supports the NcamLens LiveLink subject it can be used with the Ncam SDK Lite.

Legacy Composure

Compatible LiveLink Subjects: NcamCamera

The older, legacy Composure workflow is a little more complicated to set up than the "Composure with LiveLinkLens" workflow however it has fewer restrictions on the type of lens that is used. One consideration is that it can't be used with the NcamLens LiveLink source.

Ncam AR Suite

Compatible LiveLink Subjects: NcamCamera

The Ncam AR Suite is designed to be a plug-n-play solution for rendering virtual overlay and virtual studio composites. Requiring only a camera to be dropped into the scene, it is easy to set-up. It includes a range of useful rendering features such as hold-outs (shadow receivers) and 3D garbage mattes. One consideration is that it can't be used with the NcamLens LiveLink source.

Selecting a Workflow

The flow chart below aims to break down the options and highlight the documentation that you should read for each set-up.



Configure: nDisplay Workflow

Configure: AR Workflow 

Compositing: AR Suite

Compositing: Legacy Composure Guide

Compositing: Composure with LiveLinkLens