Monty.Frameworks.Actions.ActionSamplers (ExTbpMonty.Core v0.1.0)

Making actions from a random stream: the choices a policy draws.

Mirrors tbp/monty/frameworks/actions/action_samplers.py, which holds three names:

...ActionSamplers.ActionSampler — ported: the base, which picks one of its configured actions and asks its own sample_<name> for it;
...ActionSamplers.ConstantSampler — ported: fixed parameters, so every action of a kind is identical;
...ActionSamplers.UniformlyDistributedSampler — ported: parameters drawn uniformly from configured ranges.

How the random stream reaches a sampler

Monty passes a mutable RandomState into every sample call and mutates it in place, so consecutive draws consume one advancing stream across the whole episode. Elixir has no mutable stream, so a sampler answers the state it leaves behind and the caller threads it:

{action, rng} = ActionSampler.sample(sampler, agent_id, rng)

Threading rather than rederiving the stream from the step is the decision of docs/monty-differences.md §2.5, and it is forced by behaviour rather than chosen on cost: a policy draws several times in one step, and a stream keyed to the step alone would answer those draws identically. ActionSampler.stream/3 is where the thread begins for a policy that has none yet. scripts/bench_samplers.exs records what the threading saves.

Summary

Functions

choose(state, count)

One of the first count non-negative integers, each equally likely.

from_config(config)

The sampler a config describes.

three(state)

Three floats drawn uniformly from [0, 1), as numpy's rng.random(3).

uniform(state, low, high)

A float drawn uniformly from [low, high).

Functions

choose(state, count)

@spec choose(:rand.state(), pos_integer()) :: {non_neg_integer(), :rand.state()}

One of the first count non-negative integers, each equally likely.

from_config(config)

@spec from_config(map()) :: Monty.Frameworks.Actions.ActionSamplers.ActionSampler.t()

The sampler a config describes.

action_sampler:
  _target_: tbp.monty.frameworks.actions.action_samplers.ConstantSampler
  actions:
  - ${monty.class:tbp.monty.frameworks.actions.actions.MoveForward}
  translation_distance: 0.004

The actions are named as classes too, so they resolve the same way the sampler itself does. A parameter the sampler does not have is an error: the config asked for behaviour this port cannot give, and answering something else would make the run a different experiment.

three(state)

@spec three(:rand.state()) :: {[float()], :rand.state()}

Three floats drawn uniformly from [0, 1), as numpy's rng.random(3).

Three separate draws rather than one, because that is what the stream order then is, and a quaternion built from them must consume the stream in Monty's order for the sampler to be recognisable.

uniform(state, low, high)

@spec uniform(:rand.state(), float(), float()) :: {float(), :rand.state()}

A float drawn uniformly from [low, high).

Monty's rng.uniform(low=..., high=...).