Create a gaze-based navigation system
Kellan Cartledge looks into Unreal and how to guide a player around
in this tutorial you will learn how to create a gaze-based navigation system in unreal engine 4, using something that’s very similar to the one Kilograph used on our Suspension House experience. This tutorial will start with ue4’s first-person template and discuss how to modify this to work with a gaze-controlled system. This tutorial will also cover working in Blueprints, including creating variables, custom events and functions. We will also touch on creating custom materials with ue4.
Tracking a player’s gaze is a popular solution in video games and other interactive media as it allows you to guide the experience in a much more thoughtful way. When you know where a player is looking you can not only control their location, but trigger animations, dynamically change materials and spawn new objects. This is a big reason why we wanted to build this capability into Suspension House.
With gaze-based navigation the goal is to create a movement system that locks a viewer to a single spot, allowing them to look around but not walk like they would in a traditional first-person experience. since people will still want to move around, you need to give them clear ways to do that, which are also easy to understand. Our system works off gaze targets. They appear in front of the viewer and teleport them to a specific location after they are stared at long enough.
starting this process is a matter of using Blueprints. if this is your first time working with Blueprints in ue4 make sure to look over the unreal engine 4 documentation in order to get a more in-depth explanation of programming. 01
Use the first-person Blueprint template The first-person Blueprint template provided with unreal engine 4 is a fantastic starting point and speeds up our development process for controlling viewer movement. After launching the epic Games Launcher and creating a new Blueprint project based on the first-person template, navigate to Firstpersonbp>blueprints, and open Firstpersoncharacter. Remove the functionality from the default Firstpersoncharacter that we do not want in our experience. This would be any functionality, nodes, components and variables that allow the viewer to walk around or shoot the gun that comes with the default viewer. We only want our viewer to be able to look around the environment by using the mouse.
Once removed, you can hit the play button in the toolbar to start the experience. now the First person character is able to look around, but cannot walk.
Create the gaze target When finished, navigation targets should turn on and off to make viewer aware of where they can move in an experience. in the case of Kilograph’s Suspension House, the gaze targets were located in specific areas of the house (the kitchen, living room, balcony, stairs, bedroom).
in our tutorial these targets can be placed anywhere as long as the viewer can see them. After creating a new Blueprint Actor class named Bp_gazetarget, add two static mesh components and box collision component.
These will be used to detect the viewer’s gaze, collision with the target, and hold the icon.
Make the icon material The icon can be any image that you want for this tutorial. i’m using a simple location pin that i found online. For our case, make sure this image has an alpha channel. create a new material named M_icon. in the Material editor set it to Translucent and make sure the icon image is multiplied by a scalarparameter named Opacityamount. This will allow the opacity of the icon image to be changed as the viewer looks at the gaze target.
When you know where a player is looking you can not only control their location, but trigger animations, dynamically change materials and spawn new objects. This is a big reason why we wanted to build this capability into suspension house
Turn it into dynamic materials in order for the opacity of the icon to change as the viewer looks at the gaze target, the M_icon material must be made into a Material instance Dynamic.
Dynamic materials are used in ue4 to specify material that can be changed during an experience. To do this in our experience, in the construction script of the Bp_gazetarget create a Material instance Dynamic of M_icon.
This will allow opacity of the icon to change as the viewer gazes at the target, creating the ability for gaze targets to appear and disappear as the viewer looks around.
Blueprint Bp_gazetarget in order to make sure gaze targets (meaning the actual icon someone looks at) face the camera, we need to make a function that will rotate them toward the viewer as the viewer moves around the environment. To verify the viewer is looking at this gaze target, we check if the current gazetarget variable in the Firstperson character (this will be explained later) is this gaze target.
The seepercent variable is updated to detect how long the viewer looks at a specific gaze target. When seepercent is equal to 100 the viewer is moved to the currentgazetarget. if the viewer looks away from the currentgazetarget, seepercent is reset to 0 and the icon opacity is reset.
While this gaze target is being viewed the
Opacityamount Parameter of M_icon should be changed to the value of seepercent.
This will cause the icon image to appear and disappear as the viewer looks around the scene.
Blueprint Firstpersoncharacter in order to actually know where the viewer is looking and when the viewer is looking at a specific gaze target, we need to create a line trace from the Firstpersoncharacter camera. This line trace will be used to check if the viewer is or is not looking in the direction of a gaze target.
Back in the first-person character Blueprint, shoot a line trace from the camera location along the forward vector of the camera to a set distance.
This distance can be used to control how far from the viewer a gaze target can be detected.
if this line hits an actor, the Blueprint checks if that actor is a Bp_gazetarget, if so we set that Bp_gazetarget to the currentgazetarget variable, if not all the targets are reset in order to make sure they are invisible again.
By setting currentgazetarget to a reference of the gaze target the viewer is gazing at, we then know which icon needs to appear and which location to move the viewer to.
in order to make sure gaze targets (the actual icon someone looks at) face the camera, we need to make a function that will rotate them toward the viewer as the viewer moves around the environment. To verify the viewer is looking at this gaze target, we check if the current gazetarget variable in the Firstpersoncharacter is this gaze target
Move the player and deactivate targets now that the viewer can see the gaze targets, we need to make the player actually move to the location of the target. in Bp_gazetarget, when the seepercent variable is equal to 100, set the location of the Firstpersoncharacter to the location of the gaze target. This will move the viewer from their current location to the location of the target they were just looking at.
When the viewer is moved to the target location, the target icon needs to be disabled so the viewer does not detect it again. This is done by checking the overlap on the Box collision component and disabling the collision and Visibility of this gaze target.
Finish up Bp_gazetarget When the viewer moves on to the next target, end Overlap can be used to reactivate the old gaze target’s collision and visibility. in order to make the gaze targets disappear when the viewer is not gazing at them, we need to remember to reset the target. Do this by resetting seepercent and Opactiyamount on M_icon to 0. Then, once all of the functionality is set up it must be called every Tick so that our functions are updated and changed as the viewer looks around the environment.
Set up the experience in the experience drop a few Bp_gazetargets around the level, making sure to space them out so the viewer can move around, but not too far apart that the viewer cannot reach them.
The Firstpersoncharacter should start in the centre of the environment because this was based on the first-person template. hit play and test out the navigation system by looking around the environment.
if it seems like the viewer moves too slow, the speed can be increased in the Bp_gazetarget.
Just make sure the speed is not too fast as we want our viewer to be able to easily explore the space.