# `Monty.Frameworks.Models.GraphMatching.GraphLM` [๐Ÿ”—](https://github.com/d-led/ex.tbp.monty/blob/v0.1.0/packages/ex_tbp_monty_core/lib/monty/frameworks/models/graph_matching/graph_lm.ex#L1) A learning module whose memory is a graph of what it has seen. Mirrors `tbp/monty/frameworks/models/graph_matching.py::GraphLM`, the class `DisplacementGraphLM`, `FeatureGraphLM` and `EvidenceGraphLM` are all built on in Monty. It is the part they share: a buffer of what this episode has seen, a `GraphMemory` of what past episodes left, and the bookkeeping that turns one into the other โ€” which graphs are possible, where on the object the sensor is, and which objects a graph was built from. What it does **not** decide is how a possible match is narrowed; that is `compute_possible_matches/4`, which every subclass in Monty overrides. In Monty it is a private method called on `self`, and the subclass's version replaces it. A module here composes instead of inheriting (`docs/monty-differences.md` ยง2.9), so the same function is public and takes the module it works on. ## Why the fields are the module's Monty's `__init__` sets them and its methods read them, so they are the state this module *is*: the buffer, the memory, the possible matches and their paths, the object this episode is about, and what the module concluded. A class that composes this one keeps the same shape, which is why `LoggingUtils` can read an episode's statistics off either. ## Where it deliberately differs * **`get_unique_pose_if_available/2` raises.** Monty raises `NotImplementedError`: a graph module cannot say where the object is without a matching scheme, and a subclass supplies one. A silent `nil` would read as "no pose yet", which is a different answer. * **Errors are values, not exceptions.** A percept set with no sensor location does not raise out of a step: `add_displacements/2` answers `{:error, :no_sensor_module_location}`, which the caller turns into a step that changed nothing. # `graph_id` ```elixir @type graph_id() :: String.t() ``` # `input_channel` ```elixir @type input_channel() :: String.t() ``` # `pose` ```elixir @type pose() :: [float()] ``` # `t` ```elixir @type t() :: %Monty.Frameworks.Models.GraphMatching.GraphLM{ buffer: term(), detected_object: term(), detected_pose: term(), detected_rotation_r: term(), goals: term(), graph_id_to_target: term(), graph_memory: term(), gsg: term(), has_detailed_logger: term(), learning_module_id: term(), mode: term(), possible_matches: term(), possible_paths: term(), primary_target: term(), primary_target_rotation_quat: term(), stepwise_target_object: term(), stepwise_targets_list: term(), symmetry_evidence: term(), target_to_graph_id: term(), terminal_state: term(), tolerances: term() } ``` # `terminal_state` ```elixir @type terminal_state() :: :match | :no_match | :time_out | :pose_time_out | nil ``` # `add_detailed_stats` ```elixir @spec add_detailed_stats(t(), map()) :: map() ``` Monty's `_add_detailed_stats`: nothing extra here; subclasses add their own. # `add_displacements` ```elixir @spec add_displacements(t(), [Monty.Cmp.Message.t()]) :: {:ok, [Monty.Cmp.Message.t()]} | {:error, term()} ``` Monty's `_add_displacements`: how far the sensor moved since the last percept. One displacement, from the mean of the sensor modules' locations, set on every percept โ€” a learning module's own output moves with the sensor that produced it. The first step of an episode is a zero displacement, which is what makes the first observation a starting point rather than a move. # `add_lm_processing_to_buffer_stats` ```elixir @spec add_lm_processing_to_buffer_stats(t(), boolean()) :: t() ``` Monty's `add_lm_processing_to_buffer_stats`. # `agent_moved_since_reset?` ```elixir @spec agent_moved_since_reset?(t()) :: boolean() ``` Monty's `_agent_moved_since_reset`: whether the buffer has anything in it. A module that has buffered nothing has not moved, and its first observation is the place the displacements start from. # `buffer_percepts` ```elixir @spec buffer_percepts([Monty.Cmp.Message.t()], t()) :: t() ``` The step's percepts, in the buffer โ€” twice, as Monty writes them. Once as the step's own percepts, which is what a memory update reads later, and once per displaced percept as what this module matched against. A class that composes this one runs this after its own displacement is set. # `clear_hypotheses` ```elixir @spec clear_hypotheses(t()) :: t() ``` Monty's wipe for a module the experiment does not supervise. Such a module matched on its own during the episode, so what it concluded is a guess โ€” and a guess written into memory is worse than nothing learned. `detected_object` is what gates the write, so clearing it is the whole decision; the hypotheses go with it so that the episode's statistics do not claim a match nobody made. # `collect_stats_to_save` ```elixir @spec collect_stats_to_save(t()) :: {t(), map()} ``` Monty's `collect_stats_to_save`: what this step is worth logging. # `compute_possible_matches` ```elixir @spec compute_possible_matches( t(), Monty.Context.t(), [Monty.Cmp.Message.t()], boolean() ) :: {:error, :not_implemented} ``` Monty's `_compute_possible_matches`: which observations are still possible. The base class has nothing to say โ€” Monty raises `NotImplementedError`, and every subclass in the tree answers this instead. The answer is an error rather than a guess because a graph module with no matching scheme would narrow nothing and look like one that had not decided yet. # `conclusion` ```elixir @spec conclusion(t()) :: terminal_state() ``` What this module concluded, as the model reads it. # `exploratory_step` ```elixir @spec exploratory_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()} ``` Monty's `exploratory_step`: buffer the observation, learn nothing yet. # `fixme_update_ground_truth` ```elixir @spec fixme_update_ground_truth(t()) :: t() ``` Monty's `fixme_update_ground_truth`: which objects the graph was built from. # `from_config` ```elixir @spec from_config(map(), keyword()) :: {:ok, t()} ``` Monty's `GraphLM(**config)`: a module that learns by being told, and does not match. The base class is buildable in Monty โ€” `graph_1lm.yaml` names it, and a supervised pretraining run builds it and trains with it, because a supervised episode is handed the object and pose it saw and never asks the module to narrow anything. What the base class cannot do it does *not* do: `_compute_possible_matches` raises `NotImplementedError` upstream, and `compute_possible_matches/4` here answers `{:error, :not_implemented}` โ€” which is where a config that evaluates with this module finds out, instead of at its second step. Refusing to build it was this port saying more than Monty does: it turned a module that trains perfectly well into a configuration nobody could run. # `get_all_known_object_ids` ```elixir @spec get_all_known_object_ids(t()) :: [graph_id()] ``` Monty's `get_all_known_object_ids`: everything this module's memory holds. # `get_graph` ```elixir @spec get_graph(t(), graph_id(), input_channel() | nil) :: term() ``` Monty's `get_graph`: one object's graph in one input channel. # `get_input_channels_in_graph` ```elixir @spec get_input_channels_in_graph(t(), graph_id()) :: [input_channel()] ``` Monty's `get_input_channels_in_graph`. # `get_object_scale` ```elixir @spec get_object_scale(t(), graph_id()) :: 1 ``` Monty's `get_object_scale`: no scale detection, so the object is as stored. # `get_output` ```elixir @spec get_output(t()) :: {nil, t()} ``` Monty's `get_output`: only an evidence module has a hypothesis to publish. # `get_possible_matches` ```elixir @spec get_possible_matches(t()) :: [graph_id()] ``` Monty's `get_possible_matches`: the objects still in the running. # `get_possible_paths` ```elixir @spec get_possible_paths(t()) :: %{optional(graph_id()) => term()} ``` Monty's `get_possible_paths`, for a log or a plot. # `get_unique_pose_if_available` ```elixir @spec get_unique_pose_if_available(t(), graph_id()) :: nil ``` Monty's `get_unique_pose_if_available`: the subclass's to answer. A graph module on its own has no matching scheme, so there is nothing to turn a possible path into a pose with. Two answers are honest: no path means no pose yet, and a path only a subclass can read means the class is incomplete โ€” which is said out loud rather than answered with "not yet", because the model reads that as an episode that has not finished. # `graph_memory` ```elixir @spec graph_memory(t()) :: Monty.Frameworks.Models.GraphMatching.GraphMemory.t() | :none ``` The long-term memory this module holds. # `init_from_ltm` ```elixir @spec init_from_ltm(t()) :: t() ``` Monty's `init_from_ltm`: the hypotheses the memory already supports. `GraphMemory.get_initial_hypotheses/1` answers every object it holds and no paths โ€” there is nothing to be possible about until an observation arrives. # `load_state_dict` ```elixir @spec load_state_dict(t(), map()) :: t() ``` Monty's `load_state_dict`: a memory, and the module state that follows it. # `matching_step` ```elixir @spec matching_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()} ``` Monty's `matching_step`: what the observation makes possible, and what it does not. A location-only step is nothing to match on and returns the module unchanged; a step with features narrows the possible matches, concludes `:no_match` when nothing is left, steps the goal generator and records the step's statistics. # `matching_step` ```elixir @spec matching_step([Monty.Cmp.Message.t()], Monty.Context.t(), t(), (t(), Monty.Context.t(), [ Monty.Cmp.Message.t() ], boolean() -> {:ok, t()} | {:error, term()})) :: {:ok, t()} ``` The same, narrowing with `narrow` rather than with this class's own scheme. A matching scheme is what `DisplacementGraphLM`, `FeatureGraphLM` and the evidence module each supply, so it is a callback here for the same reason the pose is: Monty inherits the method and overrides it, and a class that composes must be able to name what it composes. # `new` ```elixir @spec new(keyword()) :: t() ``` A module over an empty memory, with its own goal generator. Options are Monty's `__init__` arguments plus the two this port's modules all take: `:learning_module_id` (Monty's default is `"LM_0"`, and a config's id replaces it), `:tolerances` (what a feature has to match within, by input channel), `:has_detailed_logger`, and `:gsg` โ€” a goal generator, built per module because it holds a copy of the module it points at. # `observation_processed?` ```elixir @spec observation_processed?(t()) :: boolean() ``` Whether the last buffered observation was one this module processed. # `propose_goals` ```elixir @spec propose_goals(t()) :: {[Monty.Cmp.Goal.t()], t()} ``` Monty's `propose_goals`: what the goal generator last asked for. Only after a step the module processed, so a sensor that has moved without looking does not redirect the agent. # `propose_region` ```elixir @spec propose_region(t()) :: {Monty.Cmp.AttentionRegion.t(), t()} ``` Monty's `propose_region`: the region its goals are in. # `receive_votes` ```elixir @spec receive_votes(term(), t()) :: {:ok, t()} ``` Monty's `receive_votes`: another module ruling an object out. A module that has not been on the object this episode has no hypothesis to remove, and joining a vote against an object it never considered would be a conclusion it did not reach. # `record_stats` ```elixir @spec record_stats(t(), map()) :: t() ``` Monty's statistics for one step, written into the buffer. `extra` is for a class that composes this one: the numbers only it knows โ€” the paths a displacement match came from, say โ€” are merged over what a graph module records by itself. Monty gets the same effect by asking the subclass from inside `collect_stats_to_save`; a struct that is not this one cannot be asked, so it says instead. # `reset_stm` ```elixir @spec reset_stm(t()) :: t() ``` Monty's `reset_stm`: forget the episode, keep what was learned. # `select_features_to_use` ```elixir @spec select_features_to_use(t(), [Monty.Cmp.Message.t()]) :: %{ optional(input_channel()) => map() } ``` Monty's `_select_features_to_use`: the features this module matches on. A feature is matchable when the module has a tolerance for it, and `pose_fully_defined` is always carried because a pose that is not defined changes how the pose it came with may be read. # `send_out_vote` ```elixir @spec send_out_vote(t()) :: {term(), t()} ``` Monty's `send_out_vote`: the objects this module knows this one is not. A negative vote, so two modules on one object need not know the same objects: what one rules out, the other need not check. # `set_detected_object` ```elixir @spec set_detected_object(t(), terminal_state()) :: t() ``` Monty's `set_detected_object`: what a conclusion says the module was looking at. A conclusion that is not a recognition names a new object โ€” `new_object` โ€” which is what makes unsupervised learning possible: a training episode that matched nothing still learned something, and it becomes the next graph. # `set_experiment_mode` ```elixir @spec set_experiment_mode( t(), Monty.Frameworks.Models.AbstractMontyClasses.LearningModule.mode() ) :: t() ``` Monty's `set_experiment_mode`: whether this episode trains or evaluates. # `set_individual_ts` ```elixir @spec set_individual_ts(t(), terminal_state()) :: t() ``` Monty's `set_individual_ts`: conclude, and record the pose if it was a match. The buffer keeps the first conclusion only โ€” an episode ends once, and the first ending is the one that mattered. # `set_stepwise_target` ```elixir @spec set_stepwise_target(t(), String.t() | nil) :: t() ``` Records which object the module's sensor is on this step. # `set_target` ```elixir @spec set_target(t(), map() | nil) :: t() ``` Monty's `fixme_reset_ground_truth`: the object this episode is about. Named "fixme" upstream because a learning module cannot work the object out; the experiment knows it and passes it in. A `nil` target leaves the one this module already had, which is what an episode of a run without resetting boundaries relies on. # `set_terminal_state` ```elixir @spec set_terminal_state(t(), terminal_state()) :: t() ``` Sets the conclusion without deciding one. Monty's `_set_terminal_state`, which a run without episode boundaries uses to forget a conclusion it no longer stands by. # `state_dict` ```elixir @spec state_dict(t()) :: map() ``` Monty's `state_dict`: the memory, and which objects it was built from. Not the buffer: what an episode saw is over, and what a later one needs is what it left. # `step_goal_generator` ```elixir @spec step_goal_generator(t(), Monty.Context.t(), [Monty.Cmp.Message.t()]) :: t() ``` One step of the goal generator, pointed at the module it now belongs to. A class that composes this one runs the generator itself, because its own fields are what the generator was told about. # `supervise` ```elixir @spec supervise(t(), map(), [number()]) :: t() ``` Monty's `_pass_target_info_to_model`: the conclusion an experiment states. A pretraining experiment does not match. It knows which object it put in front of the agent, so the episode's answer is that knowledge rather than something the module worked out โ€” and the six numbers a matching episode would have concluded for itself (where on the model, where the sensor was, the turn, the scale) are the ones it is told instead. `location_on_model` is the object's place in the world the *first* time this experiment showed it, not this episode's: a later epoch may place the object somewhere else, and the graph must stay where it was learned. That map is Monty's `first_epoch_object_location`. # `update_ltm_from_stm` ```elixir @spec update_ltm_from_stm(t()) :: t() ``` Monty's `update_ltm_from_stm`: a training episode's observations, into memory. An evaluation episode learns nothing, and an episode that saw nothing has nothing to write โ€” which is what keeps `detected_object` the gate on both. # `update_terminal_condition` ```elixir @spec update_terminal_condition(t()) :: {t(), terminal_state()} ``` Monty's `update_terminal_condition`: has this module concluded? Three answers, in Monty's order: nothing possible is `:no_match` (and the place the buffer last held is where it was); exactly one possible object with a unique pose is `:match`; anything else is not a conclusion, and a previous `:match` is withdrawn because the evidence no longer says it. # `update_terminal_condition` ```elixir @spec update_terminal_condition(t(), (t(), graph_id() -> pose() | nil)) :: {t(), terminal_state()} ``` The same, asking `pose_of` whether it can narrow the pose. A matching scheme is what turns a possible path into a pose, so the question is a callback rather than a method: `DisplacementGraphLM` and the evidence module each answer it, and `GraphLM` on its own cannot. --- *Consult [api-reference.md](api-reference.md) for complete listing*