Monty.
Copy Markdown
View Source
Monty's model for evidence-graph matching:
tbp/monty/frameworks/models/evidence_matching/model.py.
In Monty this class is the base model with a graph memory whose learning
modules carry hypotheses and accumulate evidence for them; its __init__
builds the model out of the config's sensor modules, learning modules and the
connectivity between them, and the base class steps it.
Here the two halves are separated, which is the split this port makes
everywhere: what the model is — the sensor modules, the learning modules,
the matrices — is read from the config by Monty.Studio.Setup.model_spec/2,
and running it is the runtime's business (Monty.Runtime.Model registers the
modules, the step coordinator and the episode's random stream as processes).
This module is the name a run uses for that model, which is what makes the
class in a config the class a run actually starts.
What the model does rather than owns is here: combine_votes/2, the one piece
of _combine_votes that is arithmetic rather than message passing.
Monty.Runtime.Step.State routes votes and calls it.
A config that names a model this port has not written is not run through this
one: Monty.Studio.Setup.model_class/1 answers what a config resolves to, and
says plainly when the answer is this port's own runtime instead.
Summary
Functions
The model as something a supervisor can start.
EvidenceGraphModel._combine_votes: what a module hears, in the frame of its
own sensor.
MontyForGraphMatching.fixme_set_ground_truth: what this episode is about.
MontyBase.set_experiment_mode: whether this episode trains or evaluates.
The coordinator's state, which is where an episode's conclusion lives.
The four things a recognition policy may read off the model.
One timestep: what the modules were given and what they made of it.
Stops the model's processes.
MontySupervisedObjectPretrainingExperiment._pass_target_info_to_model.
Monty.switch_to_exploratory_step: what a recognition policy can ask for.
Monty.time_out_lms: what a recognition policy can ask for instead of stopping.
MontyBase.update_ltm: write the episode into long-term memory.
Types
@type vote() :: nil | %{ possible_states: %{optional(term()) => [Monty.Cmp.Message.t()]}, sensed_pose_rel_body: [[float()]] }
What a learning module says when it is asked to vote.
possible_states holds the hypotheses that stood out above the module's vote
threshold, one list per object; sensed_pose_rel_body is the pose of the
sensor that fed it — the location, then its three pose vectors. nil is a
module with nothing to say this step.
Functions
@spec child_spec(map()) :: Supervisor.child_spec()
The model as something a supervisor can start.
Monty's constructor builds the model; here it is spec — what the config
resolved to — and starting it is the runtime's.
@spec combine_votes(%{optional(term()) => vote()}, %{optional(term()) => [term()]}) :: %{ optional(term()) => %{optional(term()) => [Monty.Cmp.Message.t()]} }
EvidenceGraphModel._combine_votes: what a module hears, in the frame of its
own sensor.
Two learning modules look at the object through different sensors, so a vote for a place on it only means something once it has been moved between them: "if I am here, you should be there." The displacement between the sending and the receiving sensor moves the vote's location, and the rotation that takes the sending sensor's pose vectors to the receiving one's turns the vote's own pose — vote rotations are not read yet, in Monty either.
The answer is keyed by receiver, then by object, which is the shape
EvidenceGraphLM.receive_votes/2 reads. A module that did not vote this step
hears nothing, and neither does one whose sources were all silent: a module
with no opinion of its own is not a place to put other modules' opinions.
MontyForGraphMatching.fixme_set_ground_truth: what this episode is about.
The experiment is the only one that knows, and a module that is never told cannot say whether it recognised anything — so this is part of the model's public surface, not the experiment's private business.
Monty's own signature takes the world's semantic_id_to_label beside the
target, and both come from the same place: the world that chose the object also
numbered it. It is an option here rather than a second positional argument,
because a world without semantics has none to give.
@spec set_experiment_mode( atom() | String.t(), Monty.Frameworks.Models.AbstractMontyClasses.LearningModule.mode() ) :: :ok
MontyBase.set_experiment_mode: whether this episode trains or evaluates.
The experiment knows — it runs training epochs or evaluation epochs — but what an episode is working towards is the model's own business: an evaluation that recognises its object ends, a training episode switches to exploring, and a model that was told nothing does neither. Monty calls this on the model before an epoch steps, so this port does too.
@spec state(atom() | String.t()) :: Monty.Runtime.Step.State.t()
The coordinator's state, which is where an episode's conclusion lives.
@spec status(atom() | String.t()) :: Monty.Experiment.RecognitionStrategy.model()
The four things a recognition policy may read off the model.
One timestep: what the modules were given and what they made of it.
Monty's Monty.step, without the model having to be a value: the modules are
processes and the coordinator keeps the step count.
Stops the model's processes.
MontySupervisedObjectPretrainingExperiment._pass_target_info_to_model.
The modules the experiment names are told the object, the pose it was placed with and where it stood; the others have their hypotheses dropped. What a module does with that is the graph module's own business, so the model passes it on rather than deciding it.
Monty.switch_to_exploratory_step: what a recognition policy can ask for.
Monty.time_out_lms: what a recognition policy can ask for instead of stopping.
MontyBase.update_ltm: write the episode into long-term memory.
The experiment calls it once per episode, after the episode has been logged
(monty_experiment.py, post_episode); each learning module decides whether
the episode it had is one to learn from.