Monty.Frameworks.Models.Salience.SensorModule.SalienceSM (ExTbpMonty.Core v0.1.0)

A wide view that proposes where to look next.

Mirrors tbp/monty/frameworks/models/salience/sensor_module.py::SalienceSM, the sensor module five of Monty's configs give their view finder. It is not a camera: it makes no percept, and what it produces is goals for the motor system — one per pixel that is on an object, scored by how interesting that pixel is — together with the region it would attend to.

A step is three judgements in a row:

the salience strategy scores every pixel of the view (Uniform, Monty's default, scores them all alike, which leaves the rest of the step to decide);
the return inhibitor suppresses the pixels near where the sensor has recently been, so the goals stop pointing at the place already seen;
the weighted scores are dithered and rescaled into [0, 1], which is what a goal's confidence means.

A step the motor system took by itself proposes nothing and changes nothing: the sensor did not look, so it has seen nowhere it wants to go — and the goal it proposed last step is still where it wanted to go.

The dither

Monty draws it from the runtime's one rng; here it comes from the episode's one stream (Monty.Random.Stream, reached as ctx.rng), in the same order and from the same NumPy state, so a salience module dithers its scores with the numbers Monty's would (docs/monty-differences.md §2.1).

What is not here

A region is only proposed when the config gives a segmentation_strategy, and Monty's own — SLIC with merging — is still to come. A module without one proposes goals and no region, which is what Monty's does too.

Summary

Types

layer()

A module and the state it was given: a strategy, a recorder, an inhibitor.

t()

Functions

from_config(config)

The salience module a config entry describes, or why it is not this class's.

new(sensor_module_id, options \\ [])

A salience module for one sensor.

state_dict(salience_sm)

What the strategy scored, the segmentation proposed and the goals say.

Types

layer()

@type layer() :: {module(), struct()}

A module and the state it was given: a strategy, a recorder, an inhibitor.

t()

@type t() :: %Monty.Frameworks.Models.Salience.SensorModule.SalienceSM{
  goals: [Monty.Cmp.Goal.t()],
  pose: term(),
  region: Monty.Cmp.AttentionRegion.t() | nil,
  return_inhibitor: layer(),
  salience_strategy: layer(),
  segmentation_strategy: layer() | nil,
  sensor_module_id: atom() | String.t(),
  telemetry: layer()
}

Functions

from_config(config)

@spec from_config(map()) :: {:ok, t()} | {:error, term()}

The salience module a config entry describes, or why it is not this class's.

Monty's entry names the sensor and, when an experiment wants more than the defaults, a salience_strategy, a return_inhibitor, a snapshot_telemetry or a segmentation_strategy. Every shipped config takes the defaults for all four, and a strategy this port has not written is an error rather than a silently uniform view.

new(sensor_module_id, options \\ [])

@spec new(atom() | String.t(), keyword()) :: t()

A salience module for one sensor.

Options are Monty's, each a module and its config: :salience_strategy (default Strategies.Uniform), :return_inhibitor (default ReturnInhibitor), :telemetry (default Telemetry.NoopSalienceSMTelemetry) and :segmentation_strategy (default none).

state_dict(salience_sm)

@spec state_dict(t()) :: map()

What the strategy scored, the segmentation proposed and the goals say.