Monty.
Copy Markdown
View Source
A view, reshaped so that every array is addressed by pixel.
Mirrors
tbp/monty/frameworks/models/salience/on_object_observation.py::OnObjectObservation
and its function of the same name. A raw observation arrives as an image and a
flat list of points — one row per pixel, three coordinates and a semantic id —
and a salience module wants them all as grids: where each pixel is in the
world, which pixels are on the object, and how interesting those pixels are.
That reshaping is this, once, rather than in each strategy.
The middle pixel is the one width * half + half of the flat point cloud, as
Monty indexes it: location_map[row][column] is the row the transform left for
that pixel.
Where the semantic mask comes from
Monty reads the mask from the flat semantic_3d array's fourth column, not
from the depth image, so a pixel that returned a depth but is not on an object
counts as background — the difference that makes a surface agent's search look
at its object rather than at the scenery.
Summary
Functions
The view's pixels, as grids: the on-object mask, the location map, and the on-object locations and salience values in the order Monty collects them — row by row, left to right.
Types
@type location() :: [float()]
@type location_map() :: [[location()]]
@type mask() :: [[boolean()]]
@type t() :: %Monty.Frameworks.Models.Salience.OnObjectObservation{ centre_location: location() | nil, location_map: location_map(), locations: [location()], on_object_map: mask(), salience: [float()] }