Rendering Features

the rendering pipelines for the ncam ar suite has a single texture input (the film) and two outputs for the cg and composite images when the ncam workflow is run the a global list of inputs and outputs is populated depending on the configuration when a new ncam camera that uses an ncam ar suite rendering pipeline is created it's inputs and outputs are populated with those from this global list this list can be observed from the ncamarlite plug in options which are found within the project settings a user wouldn't generally need to change these options as they are configured by the workflow wizard directly ncamarsuite ncamarsuite provides two production tested cameras that are purposed for different ar scenarios each camera has a different rendering pipeline component on it that renders and produces the augmented reality composite ncvirtualoverlaycamerabp virtual overlay pipeline (cg over video); ncvirtualstudiocamerabp virtual studio pipeline (video over cg); to use one of these cameras simply locate them in the content browser of unreal navigating to the plugin folder’s content/uecamera/blueprints then drag it directly into the scene each camera is derived from the parent nctrackedcameraactor which has an additional component attached to it called the cameradrawfrustumcomponent this component is driven by the camera tracking data to visualize the camera’s view frustum supported features the tables below illustrate the visual effects which are available on the different rendering pipelines that are provided with the ncam cameras optical effects virtual overlay virtual studio notes depth of field yes yes lens distortion yes yes heat haze distortion limited yes heat haze is applied to other virtual objects but will not affect the video transparency yes yes refraction limited yes virtual objects will be correctly refracted but the video will remain unaffected lighting effects virtual overlay virtual studio notes dynamic light sources yes yes stationary light sources yes yes static (baked) light sources limited yes static lights can only affect virtual objects it will not affect the video grey shadows yes yes yes yes coloured shadows coloured shadows yes yes yes yes reflections of the video in virtual surfaces reflections of the video in virtual surfaces yes yes yes yes the rendering features such as screen space reflections, depth of field, shadows etc all work coherently together with both cg and film configuring rendering features to access and configure the various features of the compositing pipeline, first locate the actor that it is attached to in the scene, this will be one of the ncam camera blueprints the pipeline can then be found in the list of its components the images below illustrate this as well as some of the parameters that are used to configure the various effects garbage mattes when using the virtual studio camera bp from the ar suite content folder it is possible to add garbage matte objects to the scene these are typically used to extend a cg background beyond the extent of the physical green screen area of the set garbage matting is configured in the following way add a "ncvirtualstudiocamerabp" camera actor to the scene from the ncamarsuite content > uecamera > meshes > garbagematteobjects folder drag the garbage matte mesh into your level add as many meshes as required, and scale and position them accordingly ensure that the meshes are "moveable" select the ncvirtualstudiocamerabp actor and navigate to the pipeline component expand the "film" section and add the garbage matte meshes in the level to the garbage matte objects array, and enable the "enable masking" checkbox inverting the garbage matte it is possible to invert the effect of the garbage matte in this case everything outside of the garbage matte will be rendered full cg, and the garbage matte itself will be rendered as a holdout showing the camera image this can be done by changing the "masking material" in the film section to the " applyinversegarbagematte " material shadows and holdouts the virtual overlay compositing pipelines have a feature that allows shadows to be cast from virtual objects and superimposed onto the video this is achieved by placing a moveable static mesh object in the scene which is positioned and orientated to approximate the surface onto which the shadow will be cast ar suite only for example a holdout object could be placed on the ground plane to allow shadows to be cast from cg objects onto the floor these objects are referred to as “ holdout objects ” and they will not be visible in the final composite output useful holdout static mesh objects can be found under “ uecamera/meshes/holdouts ” within the ncamarsuite content folder to use a holdout first place it in the scene, position and orientate it to match the approximate location of the physical object it is representing (such as the floor) and add it to the holdout actors list of the chosen ncam compositing pipeline when using hold outs it is recommended that they fully encompass the scene so that there are no areas of transparency as they are rendered black this often means that at least two hold out objects are required, one to receive the shadows and another for blocking out the background to make the set up of the scene easier ncamarsuite provides a ncskyholdout object that can be placed to encapsulate the scene by default, the material of ncskyholdout is set to ncshadowholdoutunlit which means that it will not receive any shadows similarly, any other hold out object that is to be used for blocking and therefore shouldn’t receive shadows should also use a material with an unlit shading model, e g ncshadowholdoutunlit as shown below the hold outs which are to receive shadows for composition over the film do not need any particular material or shading model configuration to be set it is just important to have all hold outs added to the “holdout actors” list of the chosen ncam compositing pipeline, as illustrated in the image above the following images illustrate this workflow with a practical example that demonstrates the casting of shadows from multiple characters onto a hold out with slightly curved edges the image below shows how the hold outs are added to the scene firstly an ncbevelledcubeholdout is placed to receive the shadows and a second hold out, the ncskysphere , is used to mask out all remaining (black) background outside of the volume of the cube hold out the ncskysphere hold out can be scaled as large as required it’s purpose is to cover the entirety of the scene including the volume of the camera trajectory both hold out objects ensure that the shadows are cast and composited correctly onto the background image the ncbevelledcubeholdout acts as the shadow receiver and the ncskyholdout ensures that if the virtual camera moves outside of the volumethat the empty void isn’t rendered on in the composite as black this behavior is further illustrated in the images below performance considerations the ncamarsuite plugin introduces three new options to increase performance when using any of our virtual overlay compositing pipelines all of the options improve the performance of compositing shadows onto the film image these options can often be crucial when trying to reach a target frame rate especially when using complex scenes shadow quality percentage captures the shadows at a lower resolution than the full frame a shadow quality of 50 % is recommended and is set by default reducing the shadow quality increases the performance significantly compared to 100 % (full target resolution) even when set to 25 % the visual difference is mostly minimal compared to the performance gain actor to render only in shadow pass actors in this list are only rendered in the shadow pass by using this, and the option below in combination, simple proxy objects can be used to cast shadows instead of their more complex counterparts which can further improve the performance actor to ignore in shadow pass all actors in this list will not be considered for use in the shadow pass and will only be rendered in the main cg pass this option is useful to prevent an object from casting a shadow onto a holdout but not onto other cg elements color spaces ncam ar suite supports three colour spaces for sdi input and output srgb rec709 logc as the unreal engine uses srgb as it’s native colour gamut choosing srgb as the input and output colour space provides color consistency through the entire compositing pipeline this means that the colour across all viewports within the unreal editor as well as on the output sdi monitor will be same using other colour spaces such as rec709 may produce higher quality, less noisy, results on the output sdi video but at the cost of introducing subtle differences from the colour of cg objects displayed in the unreal viewport the input and output colour spaces can be configured independently if required