Monty.
Copy Markdown
View Source
A learning module that recognises objects by the displacements between what it
senses: Monty's DisplacementGraphLM.
Every step, the module knows how far the sensor moved. A graph of an object knows which nodes are joined by which displacement, so "did this object's graph have an edge like that, from where I think I am" narrows the object down without comparing a single feature. Two answers at the end: the object, and the turn between the displacements it sensed and the ones its graph predicted — which together with a scale is the object's pose.
This is the simplest learning module in Monty and the one the tutorial starts from: one sensor, one module, one object, a thousand exploratory steps.
What it composes
%Monty.Frameworks.Models.GraphMatching.GraphLM{} is the graph module
underneath — the buffer, the goal generator, the votes, the conclusion — and
this adds the three things a displacement match needs: the paths being tested,
the paths each step extends them to, and the scale factors that make the
length comparison relative. Monty inherits those from GraphLM; here they are
fields of the class that uses them, and the matching step is written out
rather than delegated, because the paths are this class's state and a callback
could not hand them back.
Not ported
Nothing of Monty's class is missing: match_attribute: "displacement" compares
the step's movement with a stored one, and "PPF" compares four numbers per
edge — the length and the three angles between it and the nodes' normals —
which is what makes the match independent of how the object is turned. A
config naming a third attribute is refused rather than matched differently.
Summary
Functions
Monty's _add_detailed_stats: the paths a match came from.
Monty's _add_displacements for this class: the step's displacement and the
point-pair features of the move.
Records whether this step's observation reached the module.
Monty's _reset_modules_buffers: drop the observations, keep the memory.
Forgets what the module concluded, keeping what it believes.
Drops the hypotheses an unsupervised module never earned.
Monty's _compute_possible_matches: what the displacement says is still possible.
What this module concluded, as the model reads it.
Monty's exploratory_step: keep the observation, match nothing yet.
Monty's fixme_update_ground_truth: which objects the graph was built from.
The module a config asks for, as Monty's __init__ reads its arguments.
The turn between two sets of vectors, Monty's get_object_rotation.
How much bigger the object is than the one in memory — Monty's get_object_scale.
Monty's get_output: a displacement module publishes no hypothesis.
Monty's get_unique_pose_if_available: where the object is, once the path is
the only one left.
The graph memory this module holds, passed to the next episode.
What the module starts an episode from: every object in memory, every node as a path, every edge as the path the first movement is matched against.
Monty's load_state_dict: a memory, and the module state that follows it.
Monty's matching_step: match this step's displacement against the paths left.
A module with an empty memory.
Whether the last buffered observation was one this module processed.
Monty's propose_goals: what the goal generator last asked for.
Monty's propose_region: the region its goals are in.
Monty's receive_votes: another module ruling an object out.
The same, with the buffer and the paths emptied first — Monty's reset_stm.
Monty's send_out_vote: the objects this module knows this one is not.
Tells the module whether this episode trains or evaluates.
Names this object without emptying the module's history.
Records which object the module's sensor is on this step.
Monty's fixme_reset_ground_truth: the object this episode is about.
Sets the conclusion without asking the module to reconsider.
Monty's state_dict: the memory, and which objects it was built from.
The supervised conclusion an experiment states, in place of matching one.
Monty's update_ltm_from_stm: a training episode's observations, into memory.
Monty's update_stats_after_vote: what a vote round left it believing.
Monty's update_terminal_condition: has this module concluded?
Types
Functions
Monty's _add_detailed_stats: the paths a match came from.
@spec add_displacements(t(), [Monty.Cmp.Message.t()]) :: {:ok, [Monty.Cmp.Message.t()]} | {:error, term()}
Monty's _add_displacements for this class: the step's displacement and the
point-pair features of the move.
The displacement is what a match is made of; the pair of features is what a point-pair-feature module would match on, and Monty computes both whatever the match attribute is. Both are set on every percept, because a learning module's own output moves with the sensor.
Records whether this step's observation reached the module.
Monty's _reset_modules_buffers: drop the observations, keep the memory.
Forgets what the module concluded, keeping what it believes.
Drops the hypotheses an unsupervised module never earned.
@spec compute_possible_matches( t(), Monty.Context.t(), [Monty.Cmp.Message.t()], boolean() ) :: t()
Monty's _compute_possible_matches: what the displacement says is still possible.
This is a prediction, not a comparison of features: for every object still possible, take the paths this episode could be on, extend each along the edges whose displacement matches what was just sensed, and predict whether the sensor is still on the object. An object whose prediction is wrong — because no edge had that displacement, so the sensor cannot be on it — is ruled out.
@spec conclusion(t()) :: Monty.Frameworks.Models.GraphMatching.GraphLM.terminal_state()
What this module concluded, as the model reads it.
@spec exploratory_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()}
Monty's exploratory_step: keep the observation, match nothing yet.
Monty's fixme_update_ground_truth: which objects the graph was built from.
The module a config asks for, as Monty's __init__ reads its arguments.
:memory is the graphs the module starts from — Monty loads a state dict into
a module it has just built, which is the same thing one step earlier. A
match_attribute this port has not written is an error rather than a
different experiment, which is what new/1 says out loud.
@spec get_object_rotation([[number()]], [[number()]], boolean()) :: {[float()], [[float()]], Monty.Geometry.Rotation.t()}
The turn between two sets of vectors, Monty's get_object_rotation.
{euler degrees, matrix, rotation}: the rotation that takes the sensed
displacements onto the modelled ones, or — with get_reverse_r — the one that
does the opposite, which turns the model to fit the senses.
Two displacements along one line do not say how far to turn about it, so the answer is one of the rotations that fit; Monty's own comment says to keep moving, and so does this — nothing here refuses a pose.
How much bigger the object is than the one in memory — Monty's get_object_scale.
Monty's get_output: a displacement module publishes no hypothesis.
Monty's get_unique_pose_if_available: where the object is, once the path is
the only one left.
[x, y, z, euler_x, euler_y, euler_z, scale] — the sensor's place on the
object, the turn between the displacements it sensed and the ones the graph
predicted, and how much bigger or smaller the object is than the one learned.
Nothing until exactly one path is possible: two paths are two places to be.
@spec graph_memory(t()) :: Monty.Frameworks.Models.GraphMatching.GraphMemory.t() | :none
The graph memory this module holds, passed to the next episode.
What the module starts an episode from: every object in memory, every node as a path, every edge as the path the first movement is matched against.
Monty's init_from_ltm. The memory is what knows the graphs; this holds the
paths, because a path is where this episode's sensor has been.
Monty's load_state_dict: a memory, and the module state that follows it.
@spec matching_step([Monty.Cmp.Message.t()], Monty.Context.t(), t()) :: {:ok, t()}
Monty's matching_step: match this step's displacement against the paths left.
A step that carried only a location is nothing to match; the rest narrow the
possible matches, and a module nothing is possible for any more concludes
:no_match.
A module with an empty memory.
:k is how many neighbours each node of a graph links to — the edges are what
a displacement is matched against, so a graph with no edges can match nothing.
:tolerance is how far an observed displacement may sit from a stored one and
still be called the same; :use_relative_len divides lengths by the first
displacement, which is scale invariance at the cost of a scale you cannot
measure. :match_attribute says which of the two comparisons a match makes:
"displacement" (the default) or "PPF", whose graphs are built with
point-pair features on their edges.
Whether the last buffered observation was one this module processed.
@spec propose_goals(t()) :: {[Monty.Cmp.Goal.t()], t()}
Monty's propose_goals: what the goal generator last asked for.
@spec propose_region(t()) :: {Monty.Cmp.AttentionRegion.t(), t()}
Monty's propose_region: the region its goals are in.
Monty's receive_votes: another module ruling an object out.
The same, with the buffer and the paths emptied first — Monty's reset_stm.
Monty's send_out_vote: the objects this module knows this one is not.
@spec set_experiment_mode( t(), Monty.Frameworks.Models.AbstractMontyClasses.LearningModule.mode() ) :: t()
Tells the module whether this episode trains or evaluates.
Names this object without emptying the module's history.
Records which object the module's sensor is on this step.
Monty's fixme_reset_ground_truth: the object this episode is about.
@spec set_terminal_state( t(), Monty.Frameworks.Models.GraphMatching.GraphLM.terminal_state() ) :: t()
Sets the conclusion without asking the module to reconsider.
Monty's state_dict: the memory, and which objects it was built from.
The supervised conclusion an experiment states, in place of matching one.
Monty's update_ltm_from_stm: a training episode's observations, into memory.
Monty's update_stats_after_vote: what a vote round left it believing.
Monty's update_terminal_condition: has this module concluded?
The decision is GraphLM's; what a displacement module adds is the pose,
which Monty records as a side effect of being asked. So it is worked out
first, here, and handed to the base as the answer to "is the pose narrowed
down" — one place that computes it, and the conclusion the same as Monty's.