Unity

introduction the aim of this section is to provide the reader with a complete understanding of how ncam's integration works with the unity editor, how to install it, use it and to also serve as a reference manual for future use it is recommended that users first follow the step by step instructions on how to install ncam's plug in (ncamar) and read the section using ncamar before proceeding alternatively, eager users can skip to the quick start guide section to begin using the ncamar plug in immediately with the demonstration scenes minimum software requirements windows 10 unity lts 2019 4 latest graphics drivers blackmagic 11 1 installing the pre requisites before getting started with the ncamar plug in for unity a couple of one time steps need to be completed first blackmagic driver ncamar supports blackmagic cards for sdi video i/o it requires a specific blackmagic driver (version 11 1) please ensure that it has been correctly installed and tested loading the ncamar package into a new or existing unity project the ncamar package needs to be imported once per project as it is not persistent from one to another this is straightforward and is achieved by launching unity click on the new button to create a new project set the location and name of the project before clicking create project once unity has launched, select assets/import package/custom package from the toolbar open select the package file ncamar release x x x unitypackage you might get the following error cs0227 unsafe code may only appear if compiling with /unsafe enable "allow 'unsafe' code" in player settings to fix this error to do this navigate to edit / project settings / player tick the box for 'allow "unsafe" code' once prompted with the contents of the package, click import quick start guide the ncamar plugin is provided with several pre made example scenes so that eager users can easily evaluate it these scenes are included within the ncamar package and can be located from within the project browser inside the assets/ncam/maps folder all of the scenes have been set up to use the ncam server emulator as their data source and can be run easily by just clicking on the play button on the toolbar above the main unity viewport (once the ncam server emulator is running) the following section describes how to set up the ncam server emulator to provide simulated tracking data go to the project browser navigate to assets/ncam/maps and open an example scene by double clicking it click the play button to run the scene using the ncam server emulator ncam server emulator the purpose of the ncam server emulator is to simulate a real ncam system, allowing us to easily replicate the data that would normally be sent from a physical ncam server it also allows us to precisely control the information that is sent and test our unity scenes using a controlled environment therefore, before continuing to investigate and run any of the example projects, first open up the ncam server emulator and click on start server this sets up the server on the local host (ip address 127 0 0 1) so that any ncam clients including the nccamera in the ncamar plugin can connect to it once any of the example projects are connected to the ncam server emulator the position, rotation, lens parameters etc of the camera inside unity can be changed from inside the emulator using ncamar ncamcubear – building a basic scene in this example, we are going to recreate the ncamcubear example project which can be found in the quick start section it demonstrates the ar use case by rendering and compositing the ncam cube over a video feed that is supplied by the ncam server emulator we will be using the ncam server emulator to simulate a real ncam system therefore, before continuing to set up your scene first open the ncam server emulator and click on start server once we have created our level we should now be able to connect to the ncam server emulator and control the position and rotation of the camera within the unity editor whilst the game is running for reference the completed scene (ncamcubear) can be found in the project browser in the assets/ncam/maps folder to start, ensure that a new blank project has been created and the ncamar package has been imported into the project (see loading the ncamar package into a new or existing unity project) the next step is to set up the scene this is quite simple, we just add our ncam cube asset and an nccamera go to the project browser and navigate to assets/ncam/meshes drag and drop an ncamcube object into the viewport to add it to the scene using the inspector panel, set the cube's xyz transform to 0 so that it is located at the world origin in the scene hierarchy panel select and delete the main camera as it will be replaced by a nccamera from the ncamar plugin go to the project browser and navigate to assets/ncam/prefabs drag and drop an nccamera object into the viewport to add it to the scene it's position isn't of any consequence but for the purpose of being explicit, set its xyz translation to 0,0, 10 respectively if you have done the above steps correctly then your scene should look like the image below the settings of the nccamera are configured to use a local running instance of the ncam server emulator by default and so providing it is running, all that is left to do is click the play button! if everything was set up correctly then the game will have started and you will be presented with a screen that shows a thumbnail or full screen rendering of the ncam cube composited over a video test pattern by playing with the position of the camera in the ncam server emulator the view inside unity should react accordingly configuring an ncamar scene in this section we will cover how a scene can be configured, what options are available and where to find them we start by discussing how the ncamcubear scene that is supplied as part of the ncamar package can be modified so that it can be used in a real world demo with a connected ncam server and camera we follow by describing how the virtual world can be offset or rotated relative to the ncam datum point using the ncorigin object ncamcubear – a real world example in order to use the ncamar plugin with real world input such as with a physical ncam server and sdi video source the scene has to be changed from the default settings conveniently, all of the options for doing this can be found in a single place on the nccamera object server settings the server section of the nccamera parameters relates to the connection of an ncam server and allows the data requested from it to be specified the default parameter for the ncam server's ip address is the local host (127 0 0 1) as it expects an instance of the ncam server emulator to be running locally but in real applications this should be set to the ip of the ncam server video settings the settings that relate to the input and output video sources and how they are interpreted can be found under the video tab of the nccamera parameters it is here that the i/o device or channel can be specified by default the i/o channel is set to sdk which indicates that the video should be received via the ncam sdk this is only useful when connecting to the ncam server emulator as in most real scenarios the video will be sent and received via an sdi interface to use the sdi input and output select the sdi channel to use (sdi) the output section of the nccamera parameters governs what will be displayed to the user in the viewport and what will be output over the sdi device intuitively, the contents of the viewport are specified using the display output parameter and the sdi output using the sdi output parameter the available options are cg only, input only and comp the functions of these options are fairly self evident although it's worth noting that in order to output a composite (comp) to either the viewport or sdi a compositing node needs to first be attached to the nccamera the controls to do this and to also specify the compositing mode are provided under the compositing tab compositing settings how the ncamar plug in is being used will dictate how the final composite should be drawn for example, when rendering a virtual overlay scene the computer generated graphics should appear over the top of the input video in the case of a virtual environment the subject being filmed is often in front of a green screen which requires the video to be keyed and then overlaid on top of the cg background the ncamar plug in supports both of these compositing modes and is controlled using the settings under the compositing tab to select the method of composure a compositing object first needs to be added to the camera this is done by simply clicking on the add button the compositing order can then be specified by changing the order parameters valid combinations are film + cg for compositing in virtual overlay mode or cg + film for the virtual studio use case using ncorigin to transform the world often the origin of the virtual world and the selected location of the ncam datum point (which is the centre of the tracking space) don't line up well for example, the camera inside the unity scene may be too far away from the asset of interest or facing in the wrong direction one solution would be to move the datum point inside ncam reality to find the desired viewpoint however this is not practical if there are multiple different levels in the world and the location of ncam needs to be changed for each of them a better solution is to always keep the ncam datum point in the same place and use an ncorigin object inside unity instead ncorigin is a simple object that can be placed into the scene to define the location, rotation and scale of ncam's datum point inside the virtual world this means that the ncam camera's position will always move relatively to it, allowing the camera to be repositioned easily by just moving the ncorigin around in order for an ncorigin to be used to transform the world it needs to be added to the scene and then linked to the nccamera this can be done by go to the project browser and navigate to assets/ncam/prefabs drag and drop an ncorigin object into the viewport to add it to the scene the location and rotation of this object will define where ncam's datum point is relative to the virtual world inside unity if placed at the origin with no rotation it will have no affect inside the scene hierarchy select the nccamera in the inspector panel for the nccamera locate the origin parameter under the ncam plugin/origin tab assign the newly created ncorigin