The sensor module that reads a 2-D model pressed onto a 3-D surface: Monty's
TwoDSensorModule.
A surface agent moves, and its patch sees a piece of that surface. This module
asks a different question from CameraSM: where on a flat version of the
surface is the sensor now, and which way does the edge under it run? The answer
is a 2-D coordinate and, when there is an edge through the patch, a fully
defined pose — which is what lets a learning module build a 2-D model of a logo
on a mug.
Unrolling the surface
A step is measured in 3-D and then unrolled: the movement is projected onto the tangent plane at the surface normal and resolved along that plane's basis. Two corrections make the result the distance along the surface:
- The arc-length correction — a chord in the tangent plane is shorter
than the arc it subtends, so each component is enlarged by
SensorProcessing.arc_length_corrected_displacement/6, using the surface's curvature in that direction and the curvature frame the sensor fitted before an edge replaced it. - The transported frame — the tangent basis is carried from step to step rather than recomputed from the current normal, so the model's axes do not jump when the surface turns. Without it, a step on a curved surface would arrive in slightly different coordinates every time and the model would smear.
The first observation of an episode has no previous location, so it defines
the origin: the sensor's world x and y become its 2-D coordinates with z
set to zero. That is Monty's own assumption, and it is only right when the
camera's image plane is aligned with world xy — a camera looking along x
would need the yz plane instead.
Edges define the pose
A 2-D sensor's pose comes from edges, not from curvature: every percept has
pose_fully_defined taken away, and the edge detector puts a frame back only
when it finds an edge near the patch's centre. An edge that is geometric — a
crease in the world rather than ink on a surface — does not count, because the
module is looking for the ink. edge_strength and coherence are reported
either way, as zero when there is no edge, so a learning module can tell "no
edge" from "not measured".
Summary
Functions
Drops what was seen, keeping the delta filter's memory and the surface's frame.
The module a config asks for, as Monty's __init__ reads its arguments.
A module for one sensor.
Everything recorded so far: the raw observations and every percept made.
Monty's step: one observation, turned into a percept with a 2-D location.
Where the sensor is, given where the agent is — Monty's update_state.
Types
@type t() :: %Monty.Frameworks.Models.TwoDSensorModule{ edge_detector: term(), features: term(), filter: term(), filter_memory: term(), noise: term(), pose: term(), previous_2d_location: term(), previous_3d_location: term(), processed_observations: term(), processor: term(), save_raw_obs: term(), sensor_module_id: term(), tangent_frame: term(), telemetry: term() }
Functions
Drops what was seen, keeping the delta filter's memory and the surface's frame.
The frame is not thrown away: it is where the model's axes are, and a new object on the same surface is still measured in them.
The module a config asks for, as Monty's __init__ reads its arguments.
A module for one sensor.
Options are Monty's: :features (required), :save_raw_obs,
:pc1_is_pc2_threshold, :is_surface_sm, :edge_detector, :noise_params and
:delta_thresholds. An :edge_detector is built when the features ask for
edge strength or coherence and none was given — a module that asked for an edge
and measured nothing would report zero forever.
Everything recorded so far: the raw observations and every percept made.
@spec step( Monty.Observations.SensorObservation.t(), boolean(), Monty.Context.t(), t() ) :: {Monty.Cmp.Message.t() | nil, t()}
Monty's step: one observation, turned into a percept with a 2-D location.
The order is Monty's and each part depends on the one before: record the raw observation if asked, extract features, take the curvature pose away and give back an edge pose only if an edge defines one, add noise, advance the 2-D position, and only then let the percept filter decide whether this percept is new enough to send.
@spec update_state(Monty.Frameworks.Models.MotorSystemState.AgentState.t(), t()) :: t()
Where the sensor is, given where the agent is — Monty's update_state.
The sensor's own position and rotation in the agent's frame, turned by the agent's rotation and added to its position.