Action masks restrict the set of actions a learned concept can select, based on the current state. Masking is is useful in applications where the set of valid actions is not always the same. For example, the brain should not be allowed to route work to a piece of equipment that is down for maintenance.

Action masks are supported for nominal action types, such as type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}.

## Usage

To specify a mask, write a mask function and reference it with the mask keyword inside the curriculum for a concept. For example:

function MaskFunction(s: ObservableState) {
# The mask function takes the input to the concept graph
# and returns a dynamic constraint on the Action type for a concept.
return constraint Action <Dynamic Type Expression>
}

graph(input: ObservableState): ActionType {
concept AConcept(input): ActionType {
curriculum {
... # other curriculum elements here
}
}
}


Mask functions receive the same input as the brain, as specified with the graph keyword. The function must return a dynamic type constraint that limits the base action type for the concept.

function MaskFunction(s: ObservableState) {
# Mask functions can use control flow, variables, etc.

# Here, we'll restrict the bin in which to place an object based on its size
if s.object_size < SmallSizeThreshold {
return constraint Action {bin: number<in s.available_small_bins>}
} else if s.object_size > LargeSizeThreshold {
return constraint Action {bin: number<in s.available_large_bins>}
} else {
# No constraint
return constraint Action {}
}
}


The return type of a mask function is parameterized by the static type being constrained: Type.DynamicType<Action>. The Inkling compiler can typically infer the required type so you do not need to explicitly define the type. All return paths must return the same parameterized type.

### Dynamic type constraints

Dynamic constraint expressions start with the keyword constraint and are followed by two type expressions:

• the static (unconstrained) type, and
• a dynamic constraint applied to the first type.

A dynamic type expression uses the existing type expression syntax but allows some subexpressions to be dynamic rather than static. For example, {choice: number<in s.valid_choices>}, or {fruit: number<mask observable_state.fruit_mask>}

There are two kinds of dynamic constraints on nominal enumerations, in and mask:

Constraint Description Example Details
in Include specified values from enumeration number<in obs.allowedValues> Values not in the enumeration are ignored.
mask Boolean value for each enumeration element number<mask obs.actionMask> Array size must match enumeration size.

Tip

To apply no constraint, use {}.

For multi-dimensional action types, the constraints are applied per field. For example, if the action type is:

type Action {
a: number<A=1,B=2,C=3>,
b: number<X=1,Y=2>
}


and the constraint is:

constraint Action {a: number<in [1,2]>, b: number<in [2]>}


the valid actions will be {a: 1, b: 2} and {a:2, b:2}.

Important

Mask constraint must always result in a non-null enumeration. If all options are masked, the brain will return a runtime error.

## Examples

### Constraint using mask keyword

type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}

type ObservableState {
x: number,
y: number,
# Booleans. If there's a wall, can't move in that direction
wallAbove: number<0,1,>,
wallBelow: number<0,1,>,
wallLeft: number<0,1,>
wallRight: number<0,1,>,
}

# Only move in allowed directions.
# Note: simulator (and real world) need to ensure that at least one direction is available.

# 1 means that action is allowed. Enumeration order in Action type is Up, Down, Left, Right.
return constraint Action {cmd: number<mask [not s.wallAbove, not s.wallBelow, not s.wallLeft, not s.wallRight]>}
}


### Constraint using in

type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}

type ObservableState {
x: number,
y: number,
# Array of allowed directions, with -1 for unused entries
allowedDirections: number<0..3 step 1>[4]
}

# Only move in allowed directions.
# Note: simulator (and real world) need to ensure that at least one direction is available.
var UpOk = 1-wallAbove

# s.allowedDirections should be an array of valid action values or placeholders
# e.g.
# [0,1,2,3,4] -- all allowed
# [0,1, -1, -1] -- only Up (0) and Down (1) allowed
return constraint Action {cmd: number<in s.allowedDirections>}
}


### Using state variables in the mask but not for learning

Sometimes the information needed for the mask is not relevant to learning. For example, consider a routing decision sending work to one of several machines in a factory. If one machine is down, no work should be sent to it, but avoiding out-of-service machines does not need to be learned as an explicit concept. Instead, the brain can learn to select which machines would be best and the mask will ensure that only working machines are chosen.


type Action {
machine: number<A=1,B=2,C=3,D=4>
}

# Mask isn't needed for learning, so leave it out
type LearningState {
# info about the job to be done by one of the machines
job_property_a: number,
job_property_b: number

# machine properties that should be used to decide which is best to use
machine_speeds: number<1..100 step 1>[4],
machine_costs: number<0..1>[4],
}

# Add the mask info to get the full state that will be passed to the brain
type ObservableState extends LearningState {
# array of 4 bools: 1 if machine is available, 0 otherwise
}

graph (input: ObservableState) {
programmed function(s: ObservableState): LearningState {
# use cast to avoid writing out all the fields one by one -- works if LearningState is a subset of ObservableState
return LearningState(s)
}
}