Events
Nov 19, 11 PM - Nov 21, 11 PM
Gain the competitive edge you need with powerful AI and Cloud solutions by attending Microsoft Ignite online.
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Solvers are components that facilitate calculating an object's position and orientation according to a predefined algorithm. Example: placing an object on the surface that the user's gaze raycast intersects with.
The Solver system deterministically defines an order of operations for these transform calculations as there's no reliable way to specify to Unity the update order for components.
Solvers offer a range of behaviors to attach objects to other objects or systems. One other example would be a tag-along object that hovers in front of the user based on the camera. A solver could also be attached to a controller and an object to make the object tag along the controller. All solvers can be safely stacked--for example, a tag-along behavior plus surface magnetism plus momentum.
The Solver system consists of three categories of scripts:
Solver
: The abstract base class that all solvers derive from. It provides state tracking, smoothing parameters and implementation, automatic solver system integration, and update order.SolverHandler
: Sets the reference object tracking against (example: the main camera transform, hand ray, etc.), handles the gathering of solver components, and executes updating them in the proper order.The third category is the solver itself. The following solvers provide the building blocks for basic behavior:
Orbital
: Locks to a specified position and offset from the referenced object.ConstantViewSize
: Scales to maintain a constant size relative to the view of the referenced object.RadialView
: Keeps the object within a view cone cast by the referenced object.Follow
: Keeps the object within a set of user-defined bounds of the referenced object.InBetween
: Keeps an object in between two tracked objects.SurfaceMagnetism
: casts rays to surfaces in the world and aligns the object to that surface.DirectionalIndicator
: Determines the position and orientation of an object as a directional indicator. From the point of reference of the SolverHandler Tracked Target, this indicator will orient towards the DirectionalTarget supplied.Momentum
: Applies acceleration/velocity/friction to simulate momentum and springiness for an object being moved by other solvers/components.HandConstraint
: Constraints object to follow hands in a region that doesn't intersect the GameObject with the hands. Useful for hand constrained interactive content such as menus, etc. This solver is intended to work with XRNode
.HandConstraintPalmUp
: Derives from HandConstraint but includes logic to test if the palm is facing the user before activation. This solver only works with XRNode
controllers and will behave just like its base class with other controller types.Overlap
: Overlaps the tracked object.To use the Solver system, add one of the components listed above to a GameObject. Since all Solvers require a SolverHandler
, one will be created automatically by Unity.
Note
Examples of how to use the Solvers system can be found in the SolverExamples.scene file.
The Tracked Target Type property of the SolverHandler
component defines the point of reference all solvers will use to calculate their algorithms. For example, a value type of Head
with a simple SurfaceMagnetism
component will result in a ray cast from the head and in the direction of the user's gaze for solving which surface is hit. Potential values for the TrackedTargetType
property are:
LinePointer
transform on a controller (that is, pointer origin on a motion controller or hand controller) pointing in the direction of the line ray
TrackedHandedness
property to select the handedness preference (that is, Left, Right, Both)TrackedHandedness
property to select the handedness preference (that is, Left, Right, Both)TrackedHandJoint
property to determine the joint transform to utilizeTransformOverride
Note
For both ControllerRay and HandJoint types, the solver handler will attempt to provide the left controller/hand transform first and then the right if the former is unavailable or unless the TrackedHandedness
property specifies otherwise.
Important
Most solvers use the forward vector of the tracked transform target supplied by the SolverHandler
. When using a Hand Joint tracked target type, the forward vector of the palm joint may point through the fingers and not through the palm. This depends on the platform supplying the hand joint data. For input simulation and Windows Mixed Reality, the up vector points up through the palm (in other words, green vector is up, blue vector is forward).
To overcome this, update the Additional Rotation property on the SolverHandler
to <90, 0, 0>. This ensures that the forward vector supplied to the solvers is pointing through the palm and outward away from the hand.
Alternatively, use the Controller Ray tracked target type to get similar behavior for pointing with hands.
It's possible to add multiple Solver
components to the same GameObject, thus chaining their algorithms. The SolverHandler
components handle updating all solvers on the same GameObject. By default, the SolverHandler
calls GetComponents<Solver>()
on Start, which will return the Solvers in the order they appear in the inspector.
Furthermore, setting the Updated Linked Transform property to true will instruct that Solver
to save its calculated position, orientation, and scale to an intermediary variable accessible by all Solvers (that is, GoalPosition
). When false, the Solver
will update the GameObject's transform directly. By saving the transform properties to an intermediary location, other Solvers can perform their calculations starting from the intermediary variable. This is because Unity doesn't allow updates to gameObject.transform to stack within the same frame.
Note
Developers can modify the order of execution of Solvers by setting the SolverHandler.Solvers
property directly.
All solvers must inherit from the abstract base class, Solver
. The primary requirements of a Solver extension involve overriding the SolverUpdate
method. In this method, developers should update the inherited GoalPosition
, GoalRotation
, and GoalScale
properties to the desired values. Furthermore, it's valuable to leverage SolverHandler.TransformTarget
as the frame of reference desired by the consumer.
The code provided below gives an example of a new Solver component called InFront
that places the attached object 2 m in front of the SolverHandler.TransformTarget
. The consumer sets SolverHandler.TrackedTargetType
as Head
, then the SolverHandler.TransformTarget
will be the camera transform, and thus this Solver will place the attached GameObject 2 m in front of the users' gaze every frame.
/// <summary>
/// InFront solver positions an object 2m in front of the tracked transform target
/// </summary>
public class InFront : Solver
{
...
public override void SolverUpdate()
{
if (SolverHandler != null && SolverHandler.TransformTarget != null)
{
var target = SolverHandler.TransformTarget;
GoalPosition = target.position + target.forward * 2.0f;
}
}
}
Every Solver component has a core set of identical properties that control the core Solver behavior.
If Smoothing is enabled, the Solver will gradually update the transform of the GameObject over time to the calculated values. Every transform component's LerpTime property determines the speed of this change. For example, a higher MoveLerpTime value will result in slower increments in movement between frames.
If MaintainScale is enabled, the Solver will utilize the GameObject's default local scale.
The Orbital
class is a tag-along component that behaves like planets in a solar system. This Solver will ensure the attached GameObject orbits around the tracked transform. Thus, if the Tracked Target Type of the SolverHandler
is set to Head
, then the GameObject will orbit around the user's head with a fixed offset applied.
Developers can modify this fixed offset to keep menus or other scene components at eye level or waist level, etc., around a user. This is done by changing the Local Offset and World Offset properties. The Orientation Type property determines the rotation applied to the object if it should maintain its original rotation or always face the camera or face whatever transform is driving its position.
The RadialView
is another tag-along component that keeps a particular portion of a GameObject within the frustum of the user's view.
The Min & Max View Degrees properties determine how much of a portion of the GameObject must always be in view.
The Min & Max Distance properties determine how far the GameObject should be kept from the user. For example, walking towards the GameObject with a Min Distance of 1 m will push the GameObject away to ensure it's never closer than 1 m to the user.
Generally, the RadialView
is used with Tracked Target Type set to Head
so that the component follows the user's gaze. However, this component can function to be kept in "view" of any Tracked Target Type.
The Follow
class positions an element in front of the tracked target relative to its local forward axis. The element can be loosely constrained (also known as "tag-along") so that it doesn't follow until the tracked target moves beyond user-defined bounds.
It works similarly to the RadialView solver, with additional controls to manage Max Horizontal & Vertical View Degrees and mechanisms to alter the object's Orientation.
The InBetween
class will keep the attached GameObject between two transforms. The GameObject's own SolverHandler
Tracked Target Type and the InBetween
component's Second Tracked Target Type property define these two transform endpoints. Generally, both types will be set to CustomOverride
and the resulting SolverHandler.TransformOverride
and InBetween.SecondTransformOverride
values are set to the two tracked endpoints.
The InBetween
component will create another SolverHandler
component at runtime based on the Second Tracked Target Type and Second Transform Override properties.
Along the line between two transforms, the PartwayOffset
defines where the object will be placed with 0.5 as halfway, 1.0 at the first transform, and 0.0 at the second transform.
The SurfaceMagnetism
works by performing a raycast against a set LayerMask of surfaces and placing the GameObject at that contact point.
The Surface Normal Offset will place the GameObject a set distance in meters away from the surface in the direction of the normal at the hit point on the surface.
Conversely, the Surface Ray Offset will place the GameObject a set distance in meters away from the surface but in the opposite direction of the raycast performed. Thus, if the raycast is the user's gaze, the GameObject will move closer along the line from the hit point on the surface to the camera.
The Orientation Mode determines the type of rotation to apply in relation to the normal on the surface.
To force the associated GameObject to stay vertical in any mode other than None, enable Keep Orientation Vertical.
Note
Use the Orientation Blend property to control the balance between rotation factors when Orientation Mode is set to Blended. A value of 0.0 will have orientation entirely driven by TrackedTarget mode, and a value of 1.0 will have orientation driven entirely by SurfaceNormal.
The Overlap
is a simple solver that will keep the object's transform at the same position and rotation as the SolverHandler's
transform target.
When adding a SurfaceMagnetism
component to a GameObject, it's important to consider the layer of the GameObject and its children, if any have colliders. The component works by performing various raycasts to determine which surface to "magnet" itself against. Suppose the solver GameObject has a collider on one of the layers listed in the MagneticSurfaces
property of SurfaceMagnetism
. In that case, the raycast will likely hit itself, resulting in the GameObject attaching to its own collider point. This odd behavior can be avoided by setting the main GameObject and all children to the Ignore Ray cast layer or modifying the MagneticSurfaces
LayerMask array appropriately.
Conversely, a SurfaceMagnetism
GameObject won't collide with surfaces on a layer not listed in the MagneticSurfaces
property. We recommend that you place all desired surfaces on a dedicated layer (that is, Surfaces) and set the MagneticSurfaces
property to just this layer. Using default or everything may result in UI components or cursors contributing to the solver.
Finally, surfaces farther than the MaxRaycastDistance
property setting will be ignored by the SurfaceMagnetism
raycasts.
The DirectionalIndicator
class is a tag-along component that orients itself to the direction of the desired point in space. It's most commonly used when the Tracked Target Type of the SolverHandler
is set to Head
. In this fashion, a UX component with the DirectionalIndicator
solver will direct a user to look at the desired point in space. This point is determined by the Directional Target property.
If the directional target is viewable by the user or whichever frame of reference is set in the SolverHandler
, then this solver will disable all Renderer
components underneath it. If not viewable, then everything will be enabled on the indicator.
The size of the indicator will shrink the closer the user is to capturing the Directional Target in their FOV.
Min Indicator Scale - The minimum scale for the indicator object
Max Indicator Scale - The maximum scale for the indicator object
Visibility Scale Factor - Multiplier to increase or decrease the FOV which determines if the Directional Target point is viewable or not
View Offset - From the viewpoint of the frame of reference (that is, camera possibly) and in the indicator direction, this property defines how far the object is from the center of the viewport.
Directional Indicator Example Scene (Assets/MRTK/Examples/Demos/Solvers/Scenes/DirectionalIndicatorSolverExample.unity)
The HandConstraint
behavior provides a solver that constrains the tracked object to a region safe for hand constrained content (such as hand UI, menus, etc.) Safe regions are considered areas that don't intersect with the hand. A derived class of HandConstraint
called HandConstraintPalmUp
is also included to demonstrate a common behavior of activating the solver-tracked object when the palm is facing the user.
See the Hand Menu documentation for examples of using the Hand Constraint solver to create hand menus.
Events
Nov 19, 11 PM - Nov 21, 11 PM
Gain the competitive edge you need with powerful AI and Cloud solutions by attending Microsoft Ignite online.
Register now