# `Monty.Runtime.Episode`
[🔗](https://github.com/d-led/ex.tbp.monty/blob/v0.1.0/packages/ex_tbp_monty_core/lib/monty/runtime/episode.ex#L1)

One episode: a model stepped against an environment until it concludes.

The loop is `MontyExperiment.run_episode_steps`, which this port also has as
`Monty.Frameworks.Experiments.MontyExperiment.run_episode_steps/2` — that is
the one a studio run goes through, because a studio run also has a caller who
may stop it and steps whose reports are recorded. This module is the same loop
for a caller that has neither, and it answers a different question: it reports
`done?` and `timed_out?` separately, so a test or a probe can say which ending
an episode got, rather than only that it ended.

What the loop does, without the experiment's data loading, epochs and loggers
— those belong to whatever is running episodes, and this module is what is left
when they are taken away:

  * clear the model's short-term memory and tell it whether the episode trains
    or evaluates;
  * step the environment and the model until the model says it is done, or
    until the step limit the experiment allows;
  * hand a training episode to long-term memory.

## The environment

An episode takes a function from the *actions of the previous step* to
`{observations, proprioceptive_state}` — Monty's `Interface.step/1`. That is
all the model needs of a world, so a recorded stream of observations, a
simulator, and a Python environment behind a port are all the same thing here.

## Why the step limit is a policy

Monty's experiment does not stop an episode by counting steps; it asks a
recognition policy (`MaxTotalSteps`, `MinEvalMaxTotalSteps`,
`ObjectRecognition`, and `AnyPolicy` over them), and the policy also decides
whether undecided modules should be timed out and whether the episode should
start exploring. So this loop asks the same question, through the same
policies: `:max_steps` is sugar for `MaxTotalSteps`, and `:policy` takes any
of them.

# `environment`

```elixir
@type environment() :: ([map()] -> {map(), map()})
```

# `mode`

```elixir
@type mode() :: Monty.Frameworks.Models.AbstractMontyClasses.LearningModule.mode()
```

# `result`

```elixir
@type result() :: %{
  steps: non_neg_integer(),
  done?: boolean(),
  timed_out?: boolean(),
  terminal_states: %{
    optional(atom()) => Monty.Runtime.Step.State.terminal_state()
  },
  infos: [map()]
}
```

# `run`

```elixir
@spec run(atom() | String.t(), environment(), keyword()) :: result()
```

Runs one episode of `model_id` against `environment`.

Options:

  * `:mode` — `:train` or `:eval`; training writes to memory at the end;
  * `:policy` — a recognition policy, as `{module, state}`; defaults to
    `MaxTotalSteps` over `:max_steps`;
  * `:max_steps` — how many steps the episode may take before it is timed out;
  * `:seed` — the episode's seed, carried in the context.

A time-out is not a failure: it is the experiment giving up on an episode, and
the model is told so, so its columns are not left open.

---

*Consult [api-reference.md](api-reference.md) for complete listing*