Monty.Frameworks.Models.MotorPolicySelectors.DistantPolicySelector (ExTbpMonty.Core v0.1.0)

Copy Markdown View Source

Three policies, chosen by who proposed the goal.

Mirrors tbp/monty/frameworks/models/motor_policy_selectors.py::DistantPolicySelector, the selector a distant agent runs. A distant agent can be asked for three different kinds of step in one run, and the sender of the goal is what says which:

  • a learning module proposed it (:gsg): the module wants a hypothesis tested, so the agent teleports — JumpToGoal — and spends the following step judging where it landed;
  • a sensor module proposed it (:sm): the sensor wants to look at something, so the agent turns — LookAtGoal;
  • nobody proposed anything: the default policy wanders.

A jump in flight takes priority over everything else, including a new learning-module goal, because the agent is half-way through an experiment it has to finish: JumpToGoal is asked first, and only if it has nothing to say does the step fall through to the other two. Of each kind, the most confident goal is the one pursued.

Why it holds policies rather than functions

SinglePolicySelector holds its policy as a five-argument function, because a step is all it needs. This one needs more: it reads the jump's status to know whether a jump is in flight, and it resets and reports on all three policies. So it holds the three policy structs and threads each one's state, which is Monty's shape too — its selector holds policy objects and calls their methods.

Summary

Functions

The step, from whichever policy the goals point at.

A selector over :jump_to_goal, :look_at_goal and :default.

Starts the next episode: every policy starts over, and the record is cleared.

Each policy's own record, keyed by which policy it is.

Types

t()

@type t() :: %Monty.Frameworks.Models.MotorPolicySelectors.DistantPolicySelector{
  default: Monty.Frameworks.Models.MotorPolicies.MotorPolicy.t(),
  is_jumping: boolean(),
  jump_to_goal: Monty.Frameworks.Models.MotorPolicies.JumpToGoal.t(),
  look_at_goal: Monty.Frameworks.Models.Salience.MotorPolicy.LookAtGoal.t(),
  selected_goals: [Monty.Cmp.Goal.t() | nil],
  selected_policies: [module()]
}

Functions

call(selector, ctx, observations, state, percept, goals)

The step, from whichever policy the goals point at.

A jump in flight is judged first; if that verdict is "no actions", the step carries on to the other two rather than ending, because there is nothing left to do about the jump and everything still to do about the goals.

new(options)

@spec new(keyword()) :: t()

A selector over :jump_to_goal, :look_at_goal and :default.

All three are required: a selector with a missing policy would have to invent a step, and inventing one is what the default policy is for.

reset(selector)

@spec reset(t()) :: t()

Starts the next episode: every policy starts over, and the record is cleared.

state_dict(selector)

@spec state_dict(t()) :: map()

Each policy's own record, keyed by which policy it is.

Monty answers the same shape, and the same reason applies: what a run wants to read afterwards is the jump that was in flight, not a summary of it.