Monty. Frameworks. Models. MotorPolicySelectors. DistantPolicySelector
(ExTbpMonty.Core v0.1.0)
Copy Markdown
View Source
Three policies, chosen by who proposed the goal.
Mirrors tbp/monty/frameworks/models/motor_policy_selectors.py::DistantPolicySelector,
the selector a distant agent runs. A distant agent can be asked for three
different kinds of step in one run, and the sender of the goal is what says
which:
- a learning module proposed it (
:gsg): the module wants a hypothesis tested, so the agent teleports —JumpToGoal— and spends the following step judging where it landed; - a sensor module proposed it (
:sm): the sensor wants to look at something, so the agent turns —LookAtGoal; - nobody proposed anything: the default policy wanders.
A jump in flight takes priority over everything else, including a new
learning-module goal, because the agent is half-way through an experiment it
has to finish: JumpToGoal is asked first, and only if it has nothing to say
does the step fall through to the other two. Of each kind, the most confident
goal is the one pursued.
Why it holds policies rather than functions
SinglePolicySelector holds its policy as a five-argument function, because a
step is all it needs. This one needs more: it reads the jump's status to know
whether a jump is in flight, and it resets and reports on all three policies.
So it holds the three policy structs and threads each one's state, which is
Monty's shape too — its selector holds policy objects and calls their methods.
Summary
Functions
The step, from whichever policy the goals point at.
A selector over :jump_to_goal, :look_at_goal and :default.
Starts the next episode: every policy starts over, and the record is cleared.
Each policy's own record, keyed by which policy it is.
Types
@type t() :: %Monty.Frameworks.Models.MotorPolicySelectors.DistantPolicySelector{ default: Monty.Frameworks.Models.MotorPolicies.MotorPolicy.t(), is_jumping: boolean(), jump_to_goal: Monty.Frameworks.Models.MotorPolicies.JumpToGoal.t(), look_at_goal: Monty.Frameworks.Models.Salience.MotorPolicy.LookAtGoal.t(), selected_goals: [Monty.Cmp.Goal.t() | nil], selected_policies: [module()] }
Functions
@spec call( t(), Monty.Context.t(), map() | nil, Monty.Frameworks.Models.MotorSystemState.t(), Monty.Cmp.Message.t() | nil, [Monty.Cmp.Goal.t()] ) :: {Monty.Frameworks.Models.MotorPolicies.MotorPolicyResult.t(), t()}
The step, from whichever policy the goals point at.
A jump in flight is judged first; if that verdict is "no actions", the step carries on to the other two rather than ending, because there is nothing left to do about the jump and everything still to do about the goals.
A selector over :jump_to_goal, :look_at_goal and :default.
All three are required: a selector with a missing policy would have to invent a step, and inventing one is what the default policy is for.
Starts the next episode: every policy starts over, and the record is cleared.
Each policy's own record, keyed by which policy it is.
Monty answers the same shape, and the same reason applies: what a run wants to read afterwards is the jump that was in flight, not a summary of it.