Set up Unreal Engine with Live FX

Overview

Live FX works with Unreal Engine in various ways, but here is the basic overview:

Live FX takes in Camera Tracking data.
Live FX forwards tracking information to Unreal Engine and moves the Virtual Camera.
Unreal Engine Texture Share SDK is used to share the image from UE to Live FX.
Live FX then uses the image with LED and Greenscreen workflows.

Live FX implements camera tracking directly and forwards it to Unreal Engine, which means that even if your camera tracking solution doesn't have their own Unreal Engine plugin, or if it is outdated, it will still work as long as it is supported in Live FX.

You can also leverage Unreal Engine's Web Remote Control to make changes to your unreal scene from other computers or tablets.

Make sure to follow each step as you go along.

1. Required Setup

Currently, you must be on a Windows machine and use Unreal Engine 5.3 for Live FX to work properly. There is a workaround for earlier versions of Unreal, make sure to contact us and let us know if you need it.

2. Download and install Live FX Plugin for Unreal Engine Plugin

Download and Install the Live FX Plugin for UE.

Download the latest Live FX Unreal Engine Plugin here: https://www.assimilatesupport.com/akb/Download51050.aspx
Unzip and place the whole Plug-in folder into your Project Plugins folder (recommended) or Engine Plugins folder.
If your project does not have a Plugins folder, you can create one. Make sure to capitalize P and spell it “Plugins”. Your folder structure should look like this:

3. Enable Plugins in UE

Go to Plugins and search for Live FX, if it’s not enabled, enable the plugin.

Go to Plugins and search for Texture Share, enable the plugin (built-in).

You can enable Web Remote Control now if you know you are going to use it later (optional).

After enabling your plugins, restart Unreal Engine.

4. Enable Live Link

Go to Window>Virtual Production and open the Live Link window.

You can also click on Window and just start searching for Live Link.

In the Live Link Window, Click on Source, and go down to LiveFX LiveLink. You can leave the settings by default if using the same computer and press Ok.
💡If you want to be able to trigger Take Recorder from Unreal, make sure to select Auto Record here. Otherwise, you can leave it unchecked. If you haven’t set up Live FX properly yet, then you will likely see a yellow dot, and you may see the message “Can’t Evaluate frame for Live FX”.
This can be because either Assimilate is not on, or it’s not set up correctly.
If you have set up Live FX, make sure that in the Camera tab, the camera is set to “Active”.
Notice the Yellow Dot, which indicates it's not set up correctly yet.

5. Save Live Link as Preset (Optional)

It’s good practice to save your Preset.

Click on Presets.
Click on Save As Preset, and give it a name.

Then in Edit>Project Settings

Search for Live Link Preset, select the “Default Live Link Preset”, and select the newly created preset.

6. Add Camera in Unreal Engine

Now let's set up the Camera in Unreal Engine.

Add a Basic>Actor
Rename this actor to something like “00_CamOffset”.

We will attach a camera with camera tracking, and this empty actor will allow us to move the camera without affecting tracking.

Add a Cinematic>Cine Camera Actor.
Give it a name, like “00_LiveFXCam”

I use the prefix 00_ so that it shows up at the top.

In the Outliner, Click and drag the Cine camera onto the Empty Actor, so that the actor becomes the parent. The camera should be indented and look like this if done correctly:
Make sure the Camera is indented under the Actor
With the LiveFXCam selected, click on the “Add Component” button. Search for “Live FX” and attach the “Live FX Runtime Camera”.
Add the Live FX Runtime Camera component to the Camera
With the LiveFXCam selected, click on the “Add Component” button.
Search for “Live Link” and attach the “Live Link Controller” component.
With the Live Link ComponentController selected,
Click on the dropdown for the Subject Representation, and select the LiveFX - Camera.

When you press play if the selected camera is not playing in the viewport, you may want to set up manually.

Optionally, instead of adding the "LiveFX_RuntimeCamera as a component to the camera, you can add it in the level blueprint, and by doing so, also allow for things like switching cameras dynamically.

You would set your blueprint up like this, drag your camera from the outliner into the level blueprint to get a reference of it, and replace the "0_Zeiss_CineCameraActor" reference with your camera:

A copy of this blueprint is provided here:

https://blueprintue.com/blueprint/dhaupwq4/

There are a few ways to work with the Texture Share image that comes from Unreal Engine.

If you are using a Camera Projection method with an Inner Frustum, you can choose the Projection Setup and then under Project Media, choose Live Capture > Unreal Texture Share, then you can follow your normal workflow.

For small walls, sometimes this is not necessary and it's preferred to not have an inner frustum at all. Here is one method to achieve that.

Create a new project and set the resolution as needed, in my case I just use 1920x1080.
In the construct, go to the Filler dropdown and add black or whichever you want.
With the new Layer selected, on the bottom left, go to “Plug-Ins”.
Select “Unreal Texture Share” and Apply on Layer.

8. Unreal Engine Live Link in Live FX

Click on the Live FX menu, then click on the Live Links Menu.
Click on Unreal Live Link. Press On.
(Optional) Click Broadcast.
Click Connect.
*IP can be 127.0.0.1 if using the local machine
**If you see numbers moving at the bottom, this is set up correctly.
If using camera tracking, make sure your tracking is set up correctly and that you press “Apply” so that it applies to your camera.
If not using camera tracking, go to the Camera tab and press “Active”
You can manually move the camera in UE, by changing the values.

9. Press Play in Unreal Engine

Go back to Unreal. Next to the Play button, click the hamburger button and select “New Editor Window (PIE)”,
Maximize this new window so it takes up the entire screen. ℹ️ The size of the New Editor Window will be the resolution in Live FX, so if it is not maximized, it will be smaller than your monitor. For example, if you resize the window so that it is the left half of your 1920x1080 screen, the resolution would be 960x1080 (960=1920/2).
Go back to Live FX (The keyboard shortcut is Alt+Tab). If all is set up, you should see the image!
If you see stretching pixels on the side, that means your comp is higher resolution than the Window you opened.
Try going to full screen with the UE scene and Alt+Tab to go back and check Assimilate. You can also change your composition resolution to be smaller to match your window size.
Notice the pixel stretching. This is when the window is not in full screen mode.
If you are using camera tracking, try physically moving your camera.
If you don't have camera tracking set up, try changing the pan from the camera menu. If all is set up correctly, you should see the scene moving in Unreal Engine.
Our scene looks and behaves as expected with Camera Tracking.