Monty. Frameworks. Models. SensorModules. ObservationProcessor
(ExTbpMonty.Core v0.1.0)
Copy Markdown
View Source
Turns one camera observation into a CMP message: a location, a surface, and the features a learning module matches on.
Mirrors tbp/monty/frameworks/models/sensor_modules.py::ObservationProcessor.
It is the pure part of CameraSM — an observation in, a percept out, no state —
which is why Monty has it as a class of its own too.
What it reads
The observation has been through the experiment's transforms, so it carries
"semantic_3d" (one row per pixel: three coordinates and the on-surface mask),
"cam_to_world", "rgba" and "depth". The middle pixel is
half + width * half of the point cloud — Monty's own arithmetic for a square
patch — and on_object is that row's mask.
Which features
:features is the list a config declares, and a feature not in it is not
computed. That is the point of declaring it: feature_weights in a hidden layer
and the tolerances in graph matching are written against exactly these names,
and a percept carrying more would be one Monty's modules never see.
:object_coverage, :min_depth and :mean_depth are taken over the pixels on
the surface rather than over the whole patch.
One difference from Monty
Monty reads features["object_coverage"] when a surface sensor's middle pixel
is off the object, and raises KeyError if the config did not ask for that
feature. Here a missing coverage counts as zero (docs/monty-differences.md §2.10).
Summary
Types
@type t() :: %Monty.Frameworks.Models.SensorModules.ObservationProcessor{ features: [String.t()], is_surface_sm: boolean(), pc1_is_pc2_threshold: float(), sensor_module_id: atom() | String.t(), surface_normal_method: Monty.Frameworks.Models.SensorModules.SurfaceNormalMethod.t(), weight_curvature: boolean() }
Functions
The processor a config describes.
Options: :features and :sensor_module_id, then :pc1_is_pc2_threshold,
:surface_normal_method, :weight_curvature and :is_surface_sm, as Monty's
takes them.
@spec process(t(), map()) :: {:ok, Monty.Cmp.Message.t()} | {:error, term()}
The percept one observation makes.
{:error, :no_point_cloud} when the observation never went through the
depth-to-points transform: there is no surface to report, and inventing one
would be worse than saying so.