Monty.
Copy Markdown
View Source
An object model as a graph: one node per distinct observation, edges to the nearest neighbours.
Mirrors tbp/monty/frameworks/models/object_model.py::GraphObjectModel.
What a graph is here
Python builds a torch_geometric.data.Data, and Monty's own
torch_graph_to_numpy turns that into a bag of arrays with the same names —
which is exactly what an Elixir map is, so the port has no graph class to
invent:
pos— one[x, y, z]per node, in the model's reference frame;x— one row per node, the node id in column 0 and then every feature laid end to end;feature_mapping— feature name to{first_column, last_column}(a half-open range, as Monty'sx[:, start:end]is);norm— the surface normal of each node, when the features carry pose vectors;edge_indexandedge_attr— the neighbour links and the displacement from each edge's source to its target.
Point-pair features
A displacement module matching on match_attribute: "PPF" compares four
numbers per edge instead of a displacement, so its model carries
has_ppf: true and every build ends in add_ppf_to_graph/1 — Monty's
torch_geometric transform, translated rather than called. The four are the
length of the edge and three angles between it and the two nodes' normals,
which is what makes the comparison independent of how the object is turned.
Summary
Functions
Monty's add_ppf_to_graph: every edge's edge_attr becomes the point-pair
features of the nodes it links, replacing the displacement that was there.
Builds the graph from observations and links each node to its k_n nearest
neighbours.
The existing graph and the new observations, as one point cloud.
The index of the edge from previous_node to new_node, or nil if the
graph does not have one.
The names of the features stored in the graph, node_ids included.
The ids of the count nodes nearest each of search_locations, as bytes.
The ids of the count nodes nearest each of search_locations.
The columns of x holding one feature, as rows.
The graph itself, or nil before the model has been built.
Whether this graph stores a value for feature at every node.
A model for object_id, with no graph yet.
The number of nodes in the graph.
One [x, y, z] per node.
Replaces the graph, as when a pretrained model is loaded.
Adds new observations to the model, in the model's reference frame.
Types
Functions
Monty's add_ppf_to_graph: every edge's edge_attr becomes the point-pair
features of the nodes it links, replacing the displacement that was there.
torch_geometric's PointPairFeatures(cat=False), which is four numbers per
edge:
[ |d|, angle(n_i, d), angle(n_j, d), angle(n_i, n_j) ]where d runs from the edge's source to its target — the same four
SensorProcessing.point_pair_features/4 computes for a step's movement, so
the graph and the query cannot disagree about what a feature is.
The graph must carry norm: a normal is half of every feature, and a module
matching on point-pair features without pose vectors has nothing to compare.
Monty asserts the same (assert data.norm is not None); this raises the
sentence instead.
A graph with no edges gets an empty list rather than an error: there is no feature to compute, and a one-node model is a legitimate first observation.
Builds the graph from observations and links each node to its k_n nearest
neighbours.
k_n is asked for as edges, so the neighbour count includes the node itself —
GraphMatchingUtils.correct_k_n/2 adjusts it for small graphs — and nil
builds the point cloud without edges.
A plain graph cannot be too small for anything, so this never refuses; it
returns {:ok, model} because a grid model can, and memory treats both alike.
@spec combine_graph_information(t(), [location()], %{optional(String.t()) => list()}) :: {[location()], %{optional(String.t()) => [[float()]]}}
The existing graph and the new observations, as one point cloud.
New features become new columns, with zeros for the nodes that never had them — which is why a feature that the graph stores but the observation lacks raises: padding that with zeros would silently claim the sensor saw nothing.
@spec edge_index_between(t(), non_neg_integer(), non_neg_integer()) :: non_neg_integer() | nil
The index of the edge from previous_node to new_node, or nil if the
graph does not have one.
The names of the features stored in the graph, node_ids included.
The ids of the count nodes nearest each of search_locations, as bytes.
See Monty.Frameworks.Models.ObjectModel.find_nearest_neighbor_ids/3: this
is find_nearest_neighbors/4 with the decoding left to the reader, for a
kernel that wanted the raw answers.
The ids of the count nodes nearest each of search_locations.
One list per search location, nearest first, ties going to the lower node id so
that two runs of the same episode choose the same nodes. Monty asks its grid
model this question through a KDTree; a graph answers it through the same
structure, built over its positions for the call — one per step, however many
hypotheses ask it something.
The columns of x holding one feature, as rows.
Monty hands out x[:, start:end]; this hands out the same thing as a list of
rows, so a caller never has to know the column offsets of anything else.
The graph itself, or nil before the model has been built.
Whether this graph stores a value for feature at every node.
A model for object_id, with no graph yet.
:k_n and :graph_delta_thresholds are how the graph will be built, so they
are the model's rather than the build's: memory knows both when it makes the
model, and a model extended later must link new nodes the way it linked the
old ones. :has_ppf says the edges carry point-pair features instead of
displacements, which is a fact about the module that owns the graph — a
displacement module matching on "PPF" sets it.
The number of nodes in the graph.
One [x, y, z] per node.
Replaces the graph, as when a pretrained model is loaded.
A point-pair model whose graph arrives without them gets them here: Monty's
DisplacementGraphMemory.load_state_dict says the same thing for a memory
read from a pickle (if model[input_channel].has_ppf is False: model[...].add_ppf_to_graph()), and a loaded graph has no PPF edges of its
own.
@spec update_model( t(), [location()], %{optional(String.t()) => list()}, location(), location(), Monty.Geometry.Rotation.t() ) :: {:ok, t()}
Adds new observations to the model, in the model's reference frame.
The new observations arrive in the body's frame; location_rel_model and
object_location_rel_body are the same physical spot seen in both, and the
rotation between them is how the model reconciles the two.