The body: turns a step and the goals it was given into actions.
Mirrors tbp/monty/frameworks/models/motor_system.py, which holds three
names:
...MotorSystem.SurfacePolicyActionDetailsTelemetry— ported: what a surface policy wants remembered about the actions it chose;...MotorSystem.RuntimeMotorSystem— ported: the interface the rest of the runtime calls;...MotorSystem(this module) — ported: the implementation, which asks a policy selector and keeps the record.
The work is one sentence long: build the state from the proprioceptive state, ask the selector, remember the actions and the state they were chosen in, return the actions. Everything a policy knows lives in the selector it was built with.
Two differences from Monty, both consequences of not being able to change oneself:
- the answer comes back with the updated motor system, rather than only the actions;
- Monty's
fixme_provide_motor_systemhandout is gone: a policy that wantsmotor_only_stepset on the step says so in theMotorPolicyResultit returns, which this module then carries.
Both are recorded in docs/monty-differences.md §2.2 and §2.3.
Summary
Types
One step's worth of what was decided: the actions, and the state they were chosen in — kept together because the state explains the actions.
Functions
Everything the motor system decided, in order.
Asks the selector for this step's actions, and remembers them.
The motor system a config describes.
Whether learning modules should sit this step out.
A motor system that will ask policy_selector what to do.
Starts the next episode: no actions remembered, no telemetry, selector reset.
What the selector recorded, for a run that wants to be read afterwards.
Types
@type decided() :: {[term()], proprioceptive_state :: Monty.Frameworks.Models.MotorSystemState.ProprioceptiveState.t()}
One step's worth of what was decided: the actions, and the state they were chosen in — kept together because the state explains the actions.
@type t() :: %Monty.Frameworks.Models.MotorSystem{ action_sequence: [decided()], motor_only_step: boolean(), policy_selector: struct(), telemetry: Monty.Frameworks.Models.MotorSystem.SurfacePolicyActionDetailsTelemetry.t() }
Functions
Everything the motor system decided, in order.
@spec call( t(), Monty.Context.t(), map() | nil, Monty.Frameworks.Models.MotorSystemState.ProprioceptiveState.t(), Monty.Cmp.Message.t() | nil, [Monty.Cmp.Goal.t()] ) :: {[term()], t()}
Asks the selector for this step's actions, and remembers them.
The state is the proprioceptive state as it arrived: Monty wraps it in a
MotorSystemState, which is the same map today, so wrapping would only make
a copy that says nothing new.
The motor system a config describes.
motor_system_config:
_target_: tbp.monty.frameworks.models.motor_system.MotorSystem
policy_selector:
_target_: ...SinglePolicySelector
policy:
_target_: ...SurfacePolicyCurvatureInformedA body is its selector and its policy, so the config is the whole description of one: there is nothing here that a surface agent's config does not say.
Whether learning modules should sit this step out.
A policy sets it when it is only turning or moving, where there is nothing new for a learning module to match.
A motor system that will ask policy_selector what to do.
Monty's constructor also hands itself to the selector, for reasons its own comment calls a hack; there is nothing to hand over here.
Starts the next episode: no actions remembered, no telemetry, selector reset.
What the selector recorded, for a run that wants to be read afterwards.