Keyword: Mask
Action masks restrict the set of actions a learned concept can select, based on the current state. Masking is is useful in applications where the set of valid actions is not always the same. For example, the brain should not be allowed to route work to a piece of equipment that is down for maintenance.
Action masks are supported for nominal action types, such as
type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}.
Usage
To specify a mask, write a mask function and reference it with the mask
keyword inside the curriculum for a concept. For example:
function MaskFunction(s: ObservableState) {
# The mask function takes the input to the concept graph
# and returns a dynamic constraint on the Action type for a concept.
return constraint Action <Dynamic Type Expression>
}
graph(input: ObservableState): ActionType {
concept AConcept(input): ActionType {
curriculum {
mask MaskFunction
... # other curriculum elements here
}
}
}
Mask functions
Mask functions receive the same input as the brain, as specified with the graph
keyword. The function must return a dynamic type constraint that limits the
base action type for the concept.
function MaskFunction(s: ObservableState) {
# Mask functions can use control flow, variables, etc.
# Here, we'll restrict the bin in which to place an object based on its size
if s.object_size < SmallSizeThreshold {
return constraint Action {bin: number<in s.available_small_bins>}
} else if s.object_size > LargeSizeThreshold {
return constraint Action {bin: number<in s.available_large_bins>}
} else {
# No constraint
return constraint Action {}
}
}
The return type of a mask function is parameterized by the static type being
constrained: Type.DynamicType<Action>. The Inkling compiler can typically
infer the required type so you do not need to explicitly define the type.
All return paths must return the same parameterized type.
Dynamic type constraints
Dynamic constraint expressions start with the keyword constraint and are
followed by two type expressions:
- the static (unconstrained) type, and
- a dynamic constraint applied to the first type.
A dynamic type expression uses the existing type expression syntax but
allows some subexpressions to be dynamic rather than static. For example,
{choice: number<in s.valid_choices>}, or {fruit: number<mask observable_state.fruit_mask>}
There are two kinds of dynamic constraints on nominal enumerations, in and mask:
| Constraint | Description | Example | Details |
|---|---|---|---|
in |
Include specified values from enumeration | number<in obs.allowedValues> |
Values not in the enumeration are ignored. |
mask |
Boolean value for each enumeration element | number<mask obs.actionMask> |
Array size must match enumeration size. |
Tip
To apply no constraint, use {}.
For multi-dimensional action types, the constraints are applied per field. For example, if the action type is:
type Action {
a: number<A=1,B=2,C=3>,
b: number<X=1,Y=2>
}
and the constraint is:
constraint Action {a: number<in [1,2]>, b: number<in [2]>}
the valid actions will be {a: 1, b: 2} and {a:2, b:2}.
Important
Mask constraint must always result in a non-null enumeration. If all options are masked, the brain will return a runtime error.
Examples
Constraint using mask keyword
type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}
type ObservableState {
x: number,
y: number,
# Booleans. If there's a wall, can't move in that direction
wallAbove: number<0,1,>,
wallBelow: number<0,1,>,
wallLeft: number<0,1,>
wallRight: number<0,1,>,
}
function MaskFunction(s: ObservableState) {
# Only move in allowed directions.
# Note: simulator (and real world) need to ensure that at least one direction is available.
# 1 means that action is allowed. Enumeration order in Action type is Up, Down, Left, Right.
return constraint Action {cmd: number<mask [not s.wallAbove, not s.wallBelow, not s.wallLeft, not s.wallRight]>}
}
Constraint using in
type Action {cmd: number<Up=0, Down=1, Left=2, Right=3>}
type ObservableState {
x: number,
y: number,
# Array of allowed directions, with -1 for unused entries
allowedDirections: number<0..3 step 1>[4]
}
function MaskFunction(s: ObservableState) {
# Only move in allowed directions.
# Note: simulator (and real world) need to ensure that at least one direction is available.
var UpOk = 1-wallAbove
# s.allowedDirections should be an array of valid action values or placeholders
# e.g.
# [0,1,2,3,4] -- all allowed
# [0,1, -1, -1] -- only Up (0) and Down (1) allowed
return constraint Action {cmd: number<in s.allowedDirections>}
}
Using state variables in the mask but not for learning
Sometimes the information needed for the mask is not relevant to learning. For example, consider a routing decision sending work to one of several machines in a factory. If one machine is down, no work should be sent to it, but avoiding out-of-service machines does not need to be learned as an explicit concept. Instead, the brain can learn to select which machines would be best and the mask will ensure that only working machines are chosen.
type Action {
machine: number<A=1,B=2,C=3,D=4>
}
# Mask isn't needed for learning, so leave it out
type LearningState {
# info about the job to be done by one of the machines
job_property_a: number,
job_property_b: number
# machine properties that should be used to decide which is best to use
machine_speeds: number<1..100 step 1>[4],
machine_costs: number<0..1>[4],
}
# Add the mask info to get the full state that will be passed to the brain
type ObservableState extends LearningState {
# array of 4 bools: 1 if machine is available, 0 otherwise
available_machine_mask: number<0,1,>[4],
}
graph (input: ObservableState) {
concept RemoveMask(input): LearningState {
programmed function(s: ObservableState): LearningState {
# use cast to avoid writing out all the fields one by one -- works if LearningState is a subset of ObservableState
return LearningState(s)
}
}
output concept ChooseMachine(RemoveMask): Action {
mask function(s: ObservableState) {
return constraint Action {machine: number<mask s.available_machine_mask>}
}
... # rest of curriculum definition
}
}