Monty.Frameworks.Models.MotorPolicySelectors.SinglePolicySelector (ExTbpMonty.Core v0.1.0)

One policy, given the goal the rest of the run believes in most.

Mirrors tbp/monty/frameworks/models/motor_policy_selectors.py::SinglePolicySelector. Several learning modules can propose several goals in one step, and a body can only pursue one: this takes the most confident and hands it over, recording which goal each step was given so a run can be read afterwards.

The policy is a five-argument function, and the last thing it is given is the chosen goal or nil — nil on a step where nothing proposed anything, which a surface policy reads as "carry on exploring".

Summary

Types

policy()

What a policy is here: a struct and the module named after it, so the step asks the policy's own module — as DistantPolicySelector does, and as Monty's MotorPolicy.__call__ does.

t()

Functions

call(selector, ctx, observations, state, percept, goals)

Asks the policy about the most confident goal of this step, and records it.

new(options)

A selector that always asks policy.

reset(selector)

Starts the next episode: the policy starts over, and the record is cleared.

state_dict(selector)

The policy's own record, and the goals this selector chose.

Types

policy()

@type policy() :: Monty.Frameworks.Models.MotorPolicies.MotorPolicy.t()

What a policy is here: a struct and the module named after it, so the step asks the policy's own module — as DistantPolicySelector does, and as Monty's MotorPolicy.__call__ does.

t()

@type t() :: %Monty.Frameworks.Models.MotorPolicySelectors.SinglePolicySelector{
  policy: policy(),
  selected_goals: [Monty.Cmp.Goal.t() | nil]
}

Functions

call(selector, ctx, observations, state, percept, goals)

@spec call(
  t(),
  Monty.Context.t(),
  term(),
  Monty.Frameworks.Models.MotorSystemState.t(),
  Monty.Cmp.Message.t() | nil,
  [Monty.Cmp.Goal.t()]
) :: {Monty.Frameworks.Models.MotorPolicies.MotorPolicyResult.t(), t()}

Asks the policy about the most confident goal of this step, and records it.

With no goals the policy is still called, with nil: standing still is a policy decision too, and a surface policy explores when nothing is asked of it.

new(options)

@spec new(keyword()) :: t()

A selector that always asks policy.

:policy is required. :selected_goals is the log, and an experiment that starts mid-run can hand one over.

reset(selector)

@spec reset(t()) :: t()

Starts the next episode: the policy starts over, and the record is cleared.

Monty's SinglePolicySelector.reset resets its policy too, and it has to: a policy mid-way through a spiral or a jump would otherwise carry it into the next object.

state_dict(selector)

@spec state_dict(t()) :: map()

The policy's own record, and the goals this selector chose.