Monty.
Copy Markdown
View Source
A model whose nodes are the busiest voxels of a grid.
Mirrors tbp/monty/frameworks/models/object_model.py::GridObjectModel. Where
GraphObjectModel keeps every observation as a node, this model first sorts
observations into a voxel grid and then keeps the max_nodes voxels that were
observed most often. That is what bounds a model in two dimensions at once:
max_size bounds the space the object may occupy — observations outside it
are what GridTooSmallError is about — and num_voxels_per_dim bounds the
resolution, so a dense surface does not turn into a graph of ten thousand
nodes.
A voxel remembers the average of everything observed inside it: a location, and each feature's average. "Average" is not one operation — a hue is a mean of angles, a pose is a mean of rotations, a boolean is a majority vote — and those differences are the reason this model asks the object-model utilities for each feature by name.
The grid is a map
numpy holds each grid as a sparse 4-D tensor of shape
(voxels, voxels, voxels, width) and reads it with to_dense()[i, j, k].
A sparse grid is a map from voxel index to content, so that is what it is
here: observation_count counts observations per voxel, location_grid
keeps each voxel's average location, and feature_grid keeps each voxel's
averaged features by name. The algorithm is unchanged — the same voxels hold
the same values — and a voxel nobody observed costs nothing.
The first observation is the centre
The grid is placed so that the first observation ever made sits in the middle voxel, and every later observation is mapped against that. An object therefore grows outwards from wherever it was first touched, which is what lets a model be built without knowing where the object is.
Averaging an update into a voxel
A voxel observed again is averaged with what it held — but, in Monty, by
averaging the two averages rather than weighting by the number of
observations, except for poses (which are merged as rotations, by count) and
object ids (where the larger count wins). That is reproduced rather than
corrected: an experiment tuned on Monty's numbers expects the same values,
and a boolean averaged this way can end up at 0.5, which Monty reads as
"defined" because it is not zero.
Summary
Functions
Builds the model from an episode's observations.
The index of the edge between two nodes, or nil: a grid model has none.
A grid model has no index to read bytes from: its search compares distances.
The nearest nodes to each search location, by straight-line distance.
The graph the model currently holds, or nil before it has been built.
A grid model's values are compared rather than stored as columns.
A model for object_id that can hold an object :max_size metres across at
:num_voxels_per_dim voxels per side, thinned to :max_nodes nodes.
Adopts a graph as this model's graph.
Adds an episode's observations to the model, in the model's own frame.
Types
@type feature_mapping() :: %{ optional(String.t()) => {non_neg_integer(), non_neg_integer()} }
@type location() :: Monty.Frameworks.Utils.SpatialArithmetics.vector()
@type t() :: %Monty.Frameworks.Models.ObjectModel.GridObjectModel{ current_feature_mapping: term(), feature_grid: term(), graph: term(), location_grid: term(), location_offset: term(), location_scale_factor: term(), location_tree: term(), max_nodes: term(), max_size: term(), num_voxels_per_dim: term(), object_id: term(), observation_count: term(), use_original_graph: term() }
Functions
@spec build_model(t(), [location()], %{optional(String.t()) => list()}) :: {:ok, t()} | {:error, Monty.Frameworks.Models.ObjectModel.GridTooSmallError.t()}
Builds the model from an episode's observations.
locations are in the model's own frame; features is what the buffer
gathered, keyed by feature name with one row per observation.
{:error, %GridTooSmallError{}} when fewer than 90% of the observations fit
the grid: an object that big needs a bigger model, and storing the part that
fits would store a lie.
@spec edge_index_between(t(), non_neg_integer(), non_neg_integer()) :: nil
The index of the edge between two nodes, or nil: a grid model has none.
A grid model has no index to read bytes from: its search compares distances.
The nearest nodes to each search location, by straight-line distance.
Monty builds a KD-tree over the graph's positions and queries it; a grid model's graph is a point cloud, so there are no edges to walk — the same search, done by comparing distances.
The graph the model currently holds, or nil before it has been built.
A grid model's values are compared rather than stored as columns.
A model for object_id that can hold an object :max_size metres across at
:num_voxels_per_dim voxels per side, thinned to :max_nodes nodes.
:use_original_graph adopts a graph as it stands, grids untouched: Monty
keeps the switch for graphs that were learned elsewhere, which are matched
against exactly as they were stored.
@spec set_graph(t(), map()) :: {:ok, t()} | {:error, Monty.Frameworks.Models.ObjectModel.GridTooSmallError.t()}
Adopts a graph as this model's graph.
A graph learned elsewhere is either kept exactly as it is (:use_original_graph)
or folded into this model's grid, which re-thins it to this model's max_nodes
and resolution.
@spec update_model( t(), [location()], %{optional(String.t()) => list()}, location(), location(), Monty.Geometry.Rotation.t() ) :: {:ok, t()} | {:error, Monty.Frameworks.Models.ObjectModel.GridTooSmallError.t()}
Adds an episode's observations to the model, in the model's own frame.
The new observations arrive in the body's frame; location_rel_model and
object_location_rel_body are the same physical spot seen in both, and the
rotation between them is how the model reconciles the two — the same
transform a graph model applies before adding nodes.