Monty.
Copy Markdown
View Source
A learning module that recognises objects by the features it senses at
locations: Monty's FeatureGraphLM.
This is the first learning module Monty had, and it is the one to read to see what recognising an object out of a graph means. A step carries a location and the features there. Memory is a graph of locations, each with the features sensed at it. The module asks two questions per graph:
- Which nodes could I be standing on? Every node whose stored features are within tolerance of what was just sensed, ignoring where they are.
- Does the move agree with a pose I could be in? For every path still possible and every rotation still possible on it, turn the observed step into the model's frame by that rotation, land on a place, and keep the path if a node near that place carries the pose the rotation predicted.
A path that survives is where the sensor would be if that pose were right.
Recognition is one object left whose path is unique and whose rotation is
unique — or, for an object that looks the same from several sides, the same
set of poses lasting required_symmetry_evidence steps.
What it composes
%Monty.Frameworks.Models.GraphMatching.GraphLM{} is the graph module
underneath: the buffer, the goal generator, the votes, the conclusion, the
memory write. This adds what only a feature match has — the poses still
possible on each path, and the symmetry evidence that lets a symmetric object
be recognised at all. The paths themselves stay in the graph module's
possible_paths, because they are what its conclusion reads.
Differences from Monty, on purpose
- A step returns a new module rather than mutating one, so the callbacks that narrow the pose hand back the module they changed.
- The nearest matching node is found by sorting the candidate nodes, not by asking a k-d tree for them in order. Monty's tree is faster on a graph of tens of thousands of nodes; this module's experiments are graphs of hundreds, and a sort is the same answer with nothing to keep consistent.
- Poses are not jittered. Monty samples
n_samplesextra rotations around each aligned pose withctx.rng.vonmises, and its only caller passes zero, so the sampling is not ported; a config asking for it would need the RNG threaded in, and no config does. - Votes are negative only. Monty's feature module also votes on the locations and rotations it still considers, and uses the other modules' such votes to prune paths. This port's model routes one kind of vote — the objects a module has ruled out — so a feature module votes like the rest and does no location pruning from other modules' votes. Monty's own comment on that path says it is not being developed further.
Summary
Functions
Monty's _add_detailed_stats: the paths, the poses and the symmetry count.
Records whether this step's observation reached the module.
Monty's _reset_modules_buffers: drop the observations, keep the memory.
Forgets what the module concluded, keeping what it believes.
Drops the hypotheses an unsupervised module never earned.
Monty's _compute_possible_matches for this class: features, and the move.
What this module concluded, as the model reads it.
Monty's exploratory_step: keep the observation, match nothing yet.
Monty's fixme_update_ground_truth: which objects the graph was built from.
The module a config asks for, as Monty's __init__ reads its arguments.
Monty's get_output: a feature module publishes no hypothesis upward.
The rotations still possible on each object, one list per path.
Monty's get_unique_pose_if_available: where the object is, once the paths and
the rotations have stopped multiplying.
The graph memory this module holds, passed to the next episode.
Monty's init_from_ltm: the hypotheses a memory gives a module at the start.
Monty's load_state_dict: a memory, and the module state that follows it.
Monty's matching_step: what this step's features and move leave possible.
A module with an empty memory.
Whether the last buffered observation was one this module processed.
Monty's _match_pose_dependent_features: does the node's pose agree.
Monty's propose_goals: what the goal generator last asked for.
Monty's propose_region: the region its goals are in.
Monty's receive_votes: another module ruling an object out.
The same, with the buffer and the hypotheses emptied first — Monty's reset_stm.
Monty's send_out_vote: the objects this module knows this one is not.
Tells the module whether this episode trains or evaluates.
Records which object the module's sensor is on this step.
Monty's fixme_reset_ground_truth: the object this episode is about.
Sets the conclusion without asking the module to reconsider.
Monty's state_dict: the memory, and which objects it was built from.
The supervised conclusion an experiment states, in place of matching one.
Monty's update_ltm_from_stm: a training episode's observations, into memory.
Monty's update_stats_after_vote: what a vote round left it believing.
Monty's update_terminal_condition: has this module concluded?
Types
@type t() :: %Monty.Frameworks.Models.FeatureLocationMatching.FeatureGraphLM{ graph_lm: term(), initial_possible_poses: term(), last_num_unique_locations: term(), last_unique_poses: term(), max_match_distance: term(), path_similarity_threshold: term(), pose_similarity_threshold: term(), possible_poses: term(), required_symmetry_evidence: term(), symmetry_evidence: term(), tolerances: term(), umbilical_num_poses: term() }
Functions
Monty's _add_detailed_stats: the paths, the poses and the symmetry count.
Records whether this step's observation reached the module.
Monty's _reset_modules_buffers: drop the observations, keep the memory.
Forgets what the module concluded, keeping what it believes.
Drops the hypotheses an unsupervised module never earned.
@spec compute_possible_matches( t(), Monty.Context.t(), [Monty.Cmp.Message.t()], boolean() ) :: t()
Monty's _compute_possible_matches for this class: features, and the move.
The move is nil on the first observation of an episode — there is nothing
behind it to have moved from — and that nil is what tells the narrowing to
look for starting places rather than to extend paths.
@spec conclusion(t()) :: Monty.Frameworks.Models.GraphMatching.GraphLM.terminal_state()
What this module concluded, as the model reads it.
@spec exploratory_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()}
Monty's exploratory_step: keep the observation, match nothing yet.
Monty's fixme_update_ground_truth: which objects the graph was built from.
The module a config asks for, as Monty's __init__ reads its arguments.
:memory is the graphs the module starts from: Monty loads a state dict into
a module it has just built, which is the same thing one step earlier.
Monty's get_output: a feature module publishes no hypothesis upward.
@spec get_possible_poses(t(), boolean()) :: %{ optional(Monty.Frameworks.Models.GraphMatching.GraphMemory.graph_id()) => list() }
The rotations still possible on each object, one list per path.
Monty's get_possible_poses: as Euler angles in degrees, each the inverse of
the stored rotation — the turn that takes the sensed frame onto the model's —
which is the form a reader of a log or a chart expects to compare against the
object's pose.
@spec get_unique_pose_if_available( t(), Monty.Frameworks.Models.GraphMatching.GraphMemory.graph_id() ) :: {t(), {:ok, map()} | :not_narrowed}
Monty's get_unique_pose_if_available: where the object is, once the paths and
the rotations have stopped multiplying.
[x, y, z, euler_x, euler_y, euler_z, scale]: the last location of the one
path left, the rotation that turns the model into what the sensor saw, and a
scale — which a feature module does not measure, so it is always 1.
The symmetry counter is evidence gathered across steps, so it comes back with the answer rather than being kept to one side.
@spec graph_memory(t()) :: Monty.Frameworks.Models.GraphMatching.GraphMemory.t() | :none
The graph memory this module holds, passed to the next episode.
Monty's init_from_ltm: the hypotheses a memory gives a module at the start.
Every object in memory is possible, every node of it is a place to have started, and no pose is ruled out — and the symmetry counter starts again, because evidence about this episode's poses says nothing about an episode that has not begun.
Monty's load_state_dict: a memory, and the module state that follows it.
@spec matching_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()}
Monty's matching_step: what this step's features and move leave possible.
A step that carried only a location is nothing to match; the rest narrow the
hypotheses, and a module nothing is possible for any more concludes
:no_match.
A module with an empty memory.
:max_match_distance is how far from a place a node may be and still be the
node the sensor arrived at. :tolerances is per input channel and per feature,
in the units that feature is sensed in — except pose vectors, which are
degrees in a config and radians everywhere in the matching, so they are
converted here, once.
:initial_possible_poses is "uniform" (try every rotation in 45° steps),
"informed" (start from the pose the sensed pose vectors imply — Monty's
default), or a list of Euler poses to try exactly.
Whether the last buffered observation was one this module processed.
@spec pose_features_match?( t(), Monty.Frameworks.Models.GraphMatching.GraphMemory.graph_id(), Monty.Frameworks.Models.GraphMatching.GraphMemory.input_channel(), non_neg_integer(), Monty.Geometry.SurfacePoseVectors.t(), boolean() ) :: boolean()
Monty's _match_pose_dependent_features: does the node's pose agree.
Two comparisons, both angles: the surface normal, and — when the sensor's pose
is fully defined — the first curvature direction, whose sign is not
meaningful, so an angle past a quarter turn is folded back (π/2 - |θ - π/2|).
@spec propose_goals(t()) :: {[Monty.Cmp.Goal.t()], t()}
Monty's propose_goals: what the goal generator last asked for.
@spec propose_region(t()) :: {Monty.Cmp.AttentionRegion.t(), t()}
Monty's propose_region: the region its goals are in.
Monty's receive_votes: another module ruling an object out.
The same, with the buffer and the hypotheses emptied first — Monty's reset_stm.
Monty's send_out_vote: the objects this module knows this one is not.
@spec set_experiment_mode( t(), Monty.Frameworks.Models.AbstractMontyClasses.LearningModule.mode() ) :: t()
Tells the module whether this episode trains or evaluates.
Records which object the module's sensor is on this step.
Monty's fixme_reset_ground_truth: the object this episode is about.
@spec set_terminal_state( t(), Monty.Frameworks.Models.GraphMatching.GraphLM.terminal_state() ) :: t()
Sets the conclusion without asking the module to reconsider.
Monty's state_dict: the memory, and which objects it was built from.
The supervised conclusion an experiment states, in place of matching one.
Monty's update_ltm_from_stm: a training episode's observations, into memory.
Monty's update_stats_after_vote: what a vote round left it believing.
@spec update_terminal_condition(t()) :: {t(), Monty.Frameworks.Models.GraphMatching.GraphLM.terminal_state()}
Monty's update_terminal_condition: has this module concluded?
The decision is GraphLM's; what a feature module adds is the pose, which
Monty records as a side effect of being asked. So it is worked out first, here,
and handed to the base as the answer to "is the pose narrowed down".