What counts as "the episode is over".
Mirrors tbp/monty/experiment/recognition_strategy.py::RecognitionStrategy. Each
learning module decides its own answer; a policy turns those answers, plus the
step counters, into the one decision the episode loop asks for:
%RecognitionResult{is_done: ..., time_out_lms: ..., start_exploring: ...}.
The model, as a policy sees it
Monty's policies are handed the model and read four things off it. Rather than
reach into the runtime's processes, a policy here is given those four things as
a map — that is what keeps the decision logic pure and comparable with Monty's
own answers (scripts/record_monty_recognition_strategy.py). The episode loop is
what builds the map, from the model it is holding.
Policies are {module, state}
Each policy is a module plus the numbers it was configured with, which is how
the rest of this port composes behaviour (Monty.Experiment.RecognitionStrategy.call/3),
and it means AnyPolicy can hold a list of them without knowing what any of
them are.
What is not here
Monty's policies log and telemeter when they decide (logger.info,
telemeter.info). The decision is what is compared; the logging is the
experiment's business, so it is not on this path.
Summary
Types
The model, as a recognition policy may read it.
A policy and the numbers it was configured with.
Callbacks
The signature every policy module implements.
Types
@type model() :: %{ is_done: boolean(), is_exploring: boolean(), matching_steps: non_neg_integer(), conclusions: [Monty.Experiment.RecognitionStatus.conclusion() | nil] }
The model, as a recognition policy may read it.
is_done is Monty's model.is_done — every learning module that matters has
concluded. conclusions is each module's answer, in module order, with nil
for a module that has not concluded.
A policy and the numbers it was configured with.
Callbacks
@callback call( model(), Monty.Experiment.RecognitionStrategy.RecognitionCounter.t(), state :: term() ) :: Monty.Experiment.RecognitionStrategy.RecognitionResult.t()
The signature every policy module implements.
state is whatever the policy's new/1 decided to keep — Monty keeps them as
instance attributes, which is the same thing without the object.
Functions
@spec call(t(), model(), Monty.Experiment.RecognitionStrategy.RecognitionCounter.t()) :: Monty.Experiment.RecognitionStrategy.RecognitionResult.t()
Applies a policy to a model and the episode's counters.
The whole reason a policy is a module and a state: this is the only place that need know the shape.
The policy a config names, built.
Takes Monty's own shape — a map with _target_ and the policy's arguments —
and answers {:ok, {module, state}} or {:error, message}. wrapped and
policies are built first, so a wrapper over a policy, or an AnyPolicy over
several, comes out of the same call.
An option the policy does not have is refused rather than ignored: a config written for a policy this port has not caught up with should say so, not run a different policy.