Two things make an Elixir run disagree with a Python one: a decision this port made, and something in Monty that looks wrong. Both are here, because anyone comparing two sets of numbers needs them together.

Where an entry suspects Monty, suspected is doing work: it says what the code does, what its own comments say it means, and what evidence there is. Someone who knows the project better may read the intent differently, and nothing here has been reported upstream.

Read this first

Two entries change a run's numbers. Read those two before you diff a run.

1. §2.1 Random streams — the one that bites.
2. §2.5 A sampler answers the stream it leaves behind — the same stream, one layer down.

Everything else changes where code lives, what an error looks like, or what happens on a path Monty itself never takes.

Looking for something specific?

Status words, used in one column of the table below:

At a glance

How the links work

Monty's code is linked to thousandbrainsproject/tbp.monty at 9130364 — the revision these entries were read from, which is why line numbers are pinned to it; the function names will outlive them. The map and the shipped config copy have moved on since (monty@760ad17, priv/monty/inputs.json), so a line number here is a line of 9130364 and not of today's checkout. This port's code is linked relative to this file. Snippets are quoted verbatim, only dedented from Monty's nesting.

1. Suspected defects in Monty

1.1 The sanity check in _compute_graph_mismatch cannot fail

In one line: it can never fail, so the one check that the two graphs are being compared in the same frame does nothing.

What's wrong. The check subtracts an expression from itself, so the difference is exactly zero and the assertion compares a value with itself.

The code. goal_generation.py:671-682

# Perform the same transformation to the estimated location (sanity check)
transformed_current_loc = top_mlh["rotation"].inv().apply(
    top_mlh["location"]
) - top_mlh["rotation"].inv().apply(top_mlh["location"])
transformed_current_loc = (
    second_mlh["rotation"].apply(transformed_current_loc)
    + second_mlh["location"]
)
assert np.all(transformed_current_loc == second_mlh["location"]), (
    "Graph transformation to 2nd object reference frame not returning correct "
    "transformed location"
)

What it should be. The comment calls this a sanity check, so it should check what the comment claims: that the transform which puts the most likely object's graph into the second object's learned frame round-trips a point — transform top_mlh["location"] into the second object's frame and back, and assert it returned.

Why it matters. It is the only check that the two graphs are being compared in one frame. Without it, a broken transform would silently make the "most disagreeing" point arbitrary, and the goal generator would jump somewhere unrelated without saying so.

What this port does. The transform is ported unchanged, as EvidenceGoalGenerator's private into_frame/3 — see evidence_goal_generator.ex — and the assertion is not recomputed, with the reason in a comment there. Nothing about the numbers changes.

1.2 np.all over Euler angles: a single-axis rotation is never a pose change

In one line: a rotation about one axis never counts as a pose change, so fewer hypothesis-testing jumps happen than the comments describe.

What's wrong. np.all over three elementwise comparisons is true only when all three Euler angles differ, so a rotation about a single axis never counts as a pose change.

The code. goal_generation.py:960-973, in _check_conditions_for_hypothesis_test:

if self.prev_top_mlhs is not None and np.all(
    top_mlh["rotation"].as_euler("xyz")
    != self.prev_top_mlhs[0]["rotation"].as_euler("xyz")
):
    logger.debug("Hypothesis jump indicated: change in most-likely rotation of MLH")
    return True

What it should be. On the reading the comments support: np.any, so that any changed angle is a changed pose. The comment above the block names exactly that case — "we didn't find the mug handle following a previous jump, and have therefore eliminated a pose" — and the TODOs above ask for an angle threshold "rather than it needing to be identical", both of which say "a difference counts". If the intent was instead "only a change of all three angles counts", the comments are what is wrong.

Why it matters. It decides when the agent spends a hypothesis-testing jump. Some pose changes that should prompt one do not, so the number of jumps, and therefore the path, differs from the documented intent.

What this port does. Reproduced exactly — correcting it here would make our runs diverge from Monty's, which is the one thing a comparison cannot afford. Pinned by the test "does not treat a turn about one axis as a new pose" in evidence_goal_generator_test.exs.

1.3 np.all on the top-two ids, stricter than its own comment

In one line: the jump condition fires only when both of the top two changes — a new runner-up under the same leader does not count.

What's wrong. The condition fires only when both ids change. A new runner-up under an unchanged leader is not a change as far as this code is concerned. The comment above says "if the identities or order … of the top two MLH changes", and the log line inside says "change or shuffle in top-two MLH IDs" — neither says that both must change.

The code. goal_generation.py:942-960

# If the identities or *order* (i.e. which one is most likely)
# of the top two MLH changes, perform a new jump test, as this
# is a reasonable heuristic for us having a new interesting
# place to test
# TODO when optimizing, consider using np.any rather than np.all, i.e. as long
# as there is any change in the top two MLH
if self.prev_top_mlhs is not None and np.all(
    [
        self.prev_top_mlhs[0]["graph_id"],
        self.prev_top_mlhs[1]["graph_id"],
    ]
    != [top_id, second_id]
):
    logger.debug(
        "Hypothesis jump indicated: change or shuffle in top-two MLH IDs"
    )
    return True

What it should be. np.any, as the TODO in the same block says — unless the intent really is "only a change of both counts", in which case the comment and the log line are what is wrong. Either way the code and its own description disagree, and only Monty's authors can say which was meant.

Why it matters. Same consequence as §1.2: fewer hypothesis-testing jumps than the comment describes. The two np.alls are one reason an Elixir run and a Python run can take different numbers of steps on the same object — unless both read the intent as I do here and the comments are the thing that is wrong.

What this port does. Reproduced, pinned by "does not jump when only one of the top two changes".

1.4 argsort(...)[-2] on a single hypothesis

In one line: with one hypothesis, [-2] raises — but a guard upstream keeps the case out of reach today.

What's wrong. With one hypothesis the evidence array has length one, and [-2] raises IndexError: index -2 is out of bounds.

The code. learning_module.py:758-763

def get_top_two_pose_hypotheses_for_graph_id(self, graph_id):
    """Return top two hypotheses for a given graph_id."""
    mlh_for_graph = self._calculate_most_likely_hypothesis(graph_id)
    second_mlh_id = np.argsort(self._hypotheses[graph_id].evidence)[-2]
    second_mlh = self._get_mlh_dict_from_id(graph_id, second_mlh_id)
    return mlh_for_graph, second_mlh

What it should be. Answer "there is no second best" rather than raise: guard the length, or sort once and take the last two when there are two.

Why it matters — and why it is latent. Its only caller is EvidenceGoalGenerator._compute_graph_mismatch, whose result is only used when focus_on_pose is set, and that flag is set only after a guard that returns early while the object still has a single hypothesis — goal_generation.py:900-904 and again at 925-930: if len(self.parent_lm._hypotheses[top_id].evidence) == 1: return False. Nothing changes the hypothesis space between that guard and the call, so the crash is out of reach today. It becomes reachable the moment those two pieces of code are separated — for example by computing goals a step later than the gating decision.

What this port does. Answers {best, nil} for a single hypothesis instead of raising, and says so in the doc of EvidenceGraphLM.get_top_two_pose_hypotheses_for_graph_id/2. An exception becomes a value the caller must handle, which is a difference in its own right — see §2.6.

1.5 first_sensor is only bound inside the loop

In one line: an agent with no sensors fails with UnboundLocalError several lines later, instead of the message the code intends.

What's wrong. first_sensor is assigned on the loop's first iteration and read after it. An agent reporting no sensors leaves it unbound, so the failure is an UnboundLocalError several lines later rather than the message the code intends.

The code. motor_policies.py:905-911 — the same pattern again at 633-639, in JumpToGoal:

for ii, current_sensor in enumerate(self._pre_jump_state.sensors):
    if ii == 0:
        first_sensor = current_sensor
    assert np.all(
        self._pre_jump_state.sensors[current_sensor].rotation
        == self._pre_jump_state.sensors[first_sensor].rotation
    ), "Sensors are not identical in pose"

What it should be. Collect the rotations first and compare the collection, so the empty case is a decision the code makes rather than a name it forgot.

Why it matters. Diagnostic only: an agent with no sensors is itself nonsense, so no numeric run is affected. It costs whoever is debugging an odd environment more time than spelling out the empty case would have.

What this port does. InformedPolicy.shared_sensor_rotation/1 asks for the shared rotation with the empty case spelled out — the identity when there are none — so the failure mode is a value rather than an unbound name.

1.6 JumpToGoal saves one name and restores another

In one line: a policy resumed mid-jump comes back with no undo actions and no memory that it was jumping.

What's wrong. The jump state is saved from _undo_actions and restored into _undo_jump_actions, which nothing in JumpToGoal reads or writes. is_jumping is saved and never restored. So a policy restored from a checkpoint has no undo actions and does not know it was mid-jump.

The code. motor_policies.py:463-478, where the two names meet:

def load_state_dict(self, memento: Memento) -> None:
    self._agent_id = memento["agent_id"]
    self._undo_action = memento["undo_action"]
    self._is_undoing_jump = memento["is_undoing_jump"]
    self._pre_jump_state = memento["pre_jump_state"]
    self._undo_jump_actions = memento["undo_jump_actions"]

def state_dict(self) -> Memento:
    return {
        "agent_id": self._agent_id,
        "undo_action": self._undo_action,
        "is_jumping": self._is_jumping,
        "pre_jump_state": self._pre_jump_state,
        "undo_jump_actions": self._undo_actions,
    }

The actions the jump computes are stored in _undo_actions (line 662) and read from there when an undo is due; _undo_jump_actions appears nowhere else in the class, and _is_undoing_jump is written by load_state_dict and read by nothing.

What it should be. One name for one thing, and every saved key restored: load into _undo_actions, restore _is_jumping, and drop _is_undoing_jump if the class does not use it. Which of the two names is the right one is Monty's call; that they differ is not.

Why it matters. Only when a run is resumed from a checkpoint — mid-jump, the undo actions are lost, so a jump that landed badly cannot be undone and the sensor stays where it landed. Everywhere else this is invisible.

What this port does. Nothing: load_state_dict/state_dict are absent, by the decision recorded in motor_policy.ex — this port's checkpoint is the studio's own record of a run, not a memento handed back to a policy. If that changes, this defect is the reason to write the checkpoint test first.

1.7 A depth of zero is read as an object a metre away

In one line: "no depth" (0) reads as "object closer than 1 m", so an agent facing nothing backs away from it, every step, forever.

What's wrong. A surface agent that cannot see the object is supposed to search for it. Before it searches, _touch_object reads the depth at the centre pixel and treats anything closer than a metre as the object to move onto. A depth image reports 0 where the camera saw nothing — Monty's own convention elsewhere: sensor_processing drops points whose depth is 0, and CameraSM reports min_depth and mean_depth over the ones that are not 0. So a view with nothing in it takes the "move onto it" branch, and the distance comes out negative, because the object distance and the sensor's height are both positive. The agent backs away from the emptiness, every step, for as long as the view stays empty.

The code. motor_policies.py:1163-1179:

depth_at_center = PositioningProcedure.depth_at_center(
    agent_id=self.agent_id,
    observations=observations,
    sensor_id=view_sensor_id,
)
if depth_at_center < 1.0:
    distance = (
        depth_at_center
        - self.desired_object_distance
        - state[self.agent_id].sensors[view_sensor_id].position[2]
    )
    ...
    return MoveForward(agent_id=self.agent_id, distance=distance)

PositioningProcedure.depth_at_center (positioning_procedures.py:76-94) returns the pixel as it is: no check for "no return".

What it should be. Decide what a depth of 0 means before comparing it with a metre — search, or ask the world for a depth it trusts. Which is Monty's call; that 0 is not a distance is not.

Why it is suspected rather than observed. Monty's experiments put a transform in front of the camera that removes the case: MissingToMaxDepth(max_depth: 1) replaces every depth of 0 with 1, and 1 is not less than 1, so the search branch runs as intended. The surf agent's own config says so — tests/conf/snapshots/base_config_10distinctobj_surf_agent.yaml:152-160. So the trap is latent: it needs a depth image that still has 0 in it — a raw sensor, or a transform chain that left MissingToMaxDepth out.

What this port does. Reproduces the branch as it is, and ports the transform that moves the trap out of reach: MissingToMaxDepth (missing_to_max_depth.ex) is built from the experiment's config and applied to observations before any module reads them (Monty.Studio.Run). Without it, a surface agent against a world whose miss is depth 0 backs away 13.5 cm per step.

1.8 The pixel grid is not a camera's pixel grid

In one line: the unprojection grid spans the image's edges rather than its pixel centres, so every point is half a pixel out.

What's wrong. A depth image is unprojected with a grid that runs from −1 to +1 over the pixels, not over the pixel centres:

x, y = np.meshgrid(
    np.linspace(-1, 1, self.w[i]), np.linspace(1, -1, self.h[i])
)

The first sample lands on the image's left edge and the last on its right one. A camera's first pixel centre is half a pixel in from the edge, so the grid is a half pixel too far left at one edge, a half pixel too far right at the other, and w / (w - 1) too wide — 64/63 for the 64-pixel patches Monty's experiments use. Everything unprojected inherits both errors: a surface point is displaced by about fx · depth / w, and the patch's middle pixel — the one a sensor module reports the surface normal and the principal curvatures at — sits 1/64 of the half-width off the sensor's own axis rather than 1/128.

The code. transforms.py:597-599 and the center_id that both SensorModule and MontyForEvidenceGraphMatching compute as half_obs_dim + obs_dim * half_obs_dim (sensor_modules.py:207-210).

What it should be. linspace(-1 + 1/w, 1 - 1/w, w) — the pixel centres — which is what the simulator that produced the image rendered.

Why it is suspected rather than observed. At Monty's surface scale the displacement is microscopic: for a patch at ten times zoom on an object 5 cm away, fx · depth / w is 78 µm and the off-axis centre is 1.6 mm out. Nothing in a Monty run would notice. It is recorded because it is visible the moment the world and the transform have to agree with each other, which is what this port's stand-in simulator does.

What this port does. Reproduces Monty's grid in the transform (depth_to_3dlocations.ex), and renders the stand-in world's image on the same grid (scene.ex) so that the two disagree about nothing. A test unprojects every pixel of a rendered patch and requires the points to land on the sphere.

1.9 The mean is divided by the count twice in the plane fit

In one line: np.mean already divides by the count, so dividing again leaves the mean's outer product in the scatter matrix.

What's wrong. surface_normal_total_least_squares builds the scatter matrix a total-least-squares plane fit is made of — the covariance of the patch's points — and subtracts the mean's outer product from the second moment. The mean it subtracts has already been averaged, and is then divided by the count again:

p_mean = 1 / n_points * np.mean(x_mat, axis=0, keepdims=True).T
m_mat = 1 / n_points * np.matmul(x_mat.T, x_mat) - np.matmul(p_mean, p_mean.T)

np.mean divides by n_points already, so m_mat is C + (1 - 1/n²) μμᵀ rather than the covariance C: the mean's outer product is under-subtracted by a factor, and the matrix is inflated along the direction of the neighbourhood's own offset from the origin. The eigenvector with the smallest eigenvalue is then not the normal of the best-fit plane in general.

The code. sensor_processing.py:437-441.

What it should be. p_mean = np.mean(x_mat, axis=0, keepdims=True).T — once.

Why it is suspected rather than observed. The neighbourhood the fit uses is already expressed relative to the middle pixel (center_neighbors subtracts it), so for a symmetric neighbourhood μ is near zero and the difference is nil. It shows when the neighbourhood is one-sided — the middle pixel near the edge of the object, so that most of the patch is masked out — and even then it is small: on a five-centimetre sphere's patch at five centimetres, with a quarter-patch neighbourhood, the two estimates differ by about a degree. Measured here with numpy against the covariance the comments describe, not from a Monty run.

What this port does. Reproduces the expression as it stands (sensor_processing.ex), so a normal estimated here and one estimated by Monty move together, and records it at the code site.

1.10 The "pose not defined" flag is written where nothing reads it

In one line: the flag lands in features, while every reader looks in morphological_features.

What's wrong. A sensor module computes whether the curvature directions are meaningful — two principal curvatures that differ by less than pc1_is_pc2_threshold leave the directions arbitrary — and reports that as the morphological feature pose_fully_defined. When it could not do the fit at all (no curvature feature was requested, or no surface normal was found) it flags the percept anyway, but writes the flag into the non-morphological features:

        else:
            # Flag that PC directions are non-meaningful for e.g. downstream motor
            # policies
            features["pose_fully_defined"] = False

So a percept can carry pose_fully_defined: False in its features and pose_fully_defined: True among its morphological features at the same time, and a policy that reads the morphological one — which is where the same name is computed and where every other reader looks — sees neither. Monty's Message.get_feature_by_name searches the non-morphological features first, so a reader going through it sees the flag; a reader indexing morphological_features does not.

The code. sensor_modules.py:339-343.

What it should be. morphological_features["pose_fully_defined"] = False, which is the dict the same quantity is written to a few lines earlier (sensor_modules.py:294).

Why it is suspected rather than observed. For the experiments this port runs — a surface agent whose config asks for principal_curvatures — the else fires only when no surface normal could be fitted, and in that case process_features_in_lm is false as well, so the percept never reaches a learning module. The disagreement is visible on a percept, not in a result.

What this port does. Reproduces it (observation_processor.ex), with a test that pins the flag where Monty puts it.

1.11 The RGB blur comment names the wrong kernel size

In one line: the docstring says OpenCV picks 6*sigma + 1; measured, it picks 8*sigma + 1.

What Monty does. GaussianBlurRGB passes (kernel_size, kernel_size) to OpenCV, and its docstring says what OpenCV does when kernel_size is left at zero:

            kernel_size: Kernel size for blur. If 0 (default), OpenCV auto-computes
                from sigma using `6*sigma + 1` rounded to nearest odd. If specified,
                must be odd.

Measured against OpenCV 4.10 on a float image, the kernel it picks is 8*sigma + 1, rounded and forced odd — sigma: 2 is seventeen pixels, not thirteen (transforms.py:290-301).

What this port does. Follows the code, because the code is what decides the numbers: GaussianBlurRGB derives round(8*sigma + 1) and forces it odd, and a test pins the sizes OpenCV answers (sigma: 1 → 9, 1.5 → 13, 2 → 17). The port's kernel was checked against cv2.GaussianBlur itself to 1.0e-7, which is float32 precision.

Why it matters little, and where it would. No config that uses this transform leaves the kernel size at zero — they give a sigma and take the default kernel — so the difference is latent for Monty's own experiments, and would only show in a config that relies on the comment's arithmetic.

1.12 Lab goes through OpenCV's 8-bit tables

In one line: uint8 in, so Lab comes back 8-bit and quantised rather than floating point.

What Monty does. vocus2/images.py converts an 8-bit RGB image to Lab and divides by 255 to bring it into [0, 1]:

    return cv2.cvtColor(image, cv2.COLOR_RGB2Lab).astype(np.float32) / 255.0

(vocus2/images.py:22-28)

Because the input is uint8, OpenCV answers CV_8U — L from 0 to 255 for 0 to 100, and a and b shifted by 128 — so the conversion quantises, and OpenCV's integer path is not the same arithmetic as the floating-point definition.

What this port does. Computes the standard chain — sRGB gamma decode, OpenCV's D65 matrix, the Lab cube-root — in floating point, then applies the same 0..255 scaling and divides by 255. Measured against OpenCV 4.10 over all 16,777,216 8-bit colours, the worst disagreement is 2.67 units of 255, in the dark saturated corner: about one percent of the range.

Why it matters little. The two opponent spaces, which are what VOCUS2's a/b channels actually use for colour opponency, are exact. Lab reaches the port only where a config names it as its colour converter, and a one-percent lightness difference moves a salience weight by about as much. Reproducing it exactly would mean reimplementing OpenCV's lookup tables, which is a different kind of promise than this port makes: it reproduces Monty's numbers, not OpenCV's rounding.

1.13 cv2's area resize does something else when only one axis enlarges

In one line: enlarge one axis and shrink the other, and INTER_AREA stops being an area average — it is neither area nor linear.

What Monty does. vocus2/images.py::resize is a thin wrapper over cv2.resize, and the pyramids use INTER_AREA to shrink by an octave and INTER_CUBIC to enlarge:

def resize(
    image: npt.NDArray[np.float32],
    shape: tuple[int, int],
    interpolation: int = cv2.INTER_AREA,
) -> npt.NDArray[np.float32]:
    return cv2.resize(image, (shape[1], shape[0]), interpolation=interpolation)

(vocus2/images.py:64-69)

Measured, INTER_AREA is the area average whenever the destination is no larger on either axis (checked in one and two dimensions). When the destination is larger on exactly one axis and smaller on the other, OpenCV answers something that is neither the area average nor the linear interpolation: a 3×8 image resized to 6×3 has 0.625 in its first pixel where the area average is 0.875.

What this port does. Answers the area average in every case, which is what the name means. The parity fixture in images_parity_test.exs pins the cases Monty reaches: an octave halving, a non-integer shrink, and three cubic enlargements.

Why it matters little, and where it would. A pyramid only ever shrinks by area and enlarges by cubic, so no Monty experiment reaches the mixed case. It would show in a strategy that resized an image to a shape larger on one axis only — none does today.

1.14 A pixel with no depth becomes the nearest patch in the view

In one line: an empty pixel scores as the nearest thing in view, so a hole in the depth map looks like an object held against the lens.

What Monty does. DepthSalience.process turns a depth image into -log(depth) and then removes the infinities:

        image = -np.log(image).astype(np.float32)
        image = np.nan_to_num(image, posinf=0.0)

(vocus2/vocus2.py:316-317)

A depth of zero — nothing there — is exactly what produces the +inf, so this line decides what an empty pixel means to the depth channel. Mapping it to zero puts it at the top of the log-depth scale: a real depth of 2 m is about -0.69 here, so an empty pixel arrives 0.69 nearer than everything else in the view, and the centre/surround difference that follows is a strong positive response. The pixel is then one of the most salient in the map.

Why it is probably not what was meant. The line is removing infinities, not choosing a meaning for absence; the natural reading of "no depth" is "no evidence", which would be a large positive log-depth — the bottom of the scale — or a mask. As written, a hole in the depth map is indistinguishable from an object held against the lens.

What this port does. Reproduces it, arithmetic and consequence, because a number that differs from Monty's is worse than a number that agrees with a suspected defect. A test pins the behaviour: the empty corner answers a positive salience, lower than a real near step's but not zero, and DepthSalience's docs say so at the code site.

Why it is latent for Monty's own experiments. The transforms a camera runs before a salience module sees its depth — MissingToMaxDepth and DepthTo3DLocations — turn a missing reading into the maximum depth, not into zero, so the case is rare in a configured run and reachable mainly by a caller that hands a depth image straight to the strategy.

1.15 Monty's two Gaussians are not the same Gaussian

In one line: cv2.GaussianBlur and scipy.ndimage.gaussian_filter differ in kernel size, edge rule and reach — they are not interchangeable.

What. The salience code blurs with OpenCV — cv2.GaussianBlur, kernel size 8σ+1 (§1.11), BORDER_REFLECT_101 (§1.11) — and the positioning code smooths with SciPy — scipy.ndimage.gaussian_filter(image, sigma, mode="constant"), a kernel of radius int(4σ + 0.5), normalised to one, applied separably with zero padding. Same mathematical object, different truncation, different edge rule, and a factor-of-two difference in how far the kernel reaches.

Why it matters to a port. Both are now implemented, and reusing one for the other would be wrong by more than the smoothing: at σ = 1 the SciPy kernel has nine taps and the OpenCV one seventeen. The SciPy rule was measured against SciPy's own answer over an image of random values — worst difference 2e-16 over σ = 1, 2, 3 — before it was written down, and the recorded answers of GetGoodView check the whole of it, because the pixel the procedure aims at is the maximum of the smoothed mask.

A plateau is not a specification. On a symmetric object the smoothed mask has a flat maximum, and which pixel NumPy's argmax returns is then a fact about summation order, not about Monty: two faithful implementations can disagree by a pixel. The fixture therefore uses a notched object, whose brightest pixel is clear of the runner-up by 0.02 of the mask's height.

2. Where this port deliberately differs

Decisions, not suspected defects. Each is a place where two runs can differ without either being wrong.

2.1 Random streams: one stream per episode, NumPy's numbers

In one line: Monty threads one mutable RNG through every module; so does this port — but as a process, not a value passed around.

The difference. Monty threads one mutable RandomState (ctx.rng) through every module in step order, so what a module draws depends on how many draws everything before it made. This port used to derive a stream per module from the episode seed, the module id and the step, and it no longer does: the episode's one stream is Monty.Random.Stream, seeded with the episode's seed and drawn by whatever the coordinator steps next.

The code. motor_policies.py:217 is where a policy draws (self.action_sampler.sample(self.agent_id, ctx.rng)). Seventeen calls read ctx.rng across eight files — motor_policies.py ten times, the sensor modules and goal_generation.py once each — so everything that samples shares one stream. Ours: Monty.Random.Stream (the stream) and Monty.Random.Numpy (the generator), reached from a module as ctx.rng.

Why a process is the faithful translation. A single mutable object shared by everything in a step is what the BEAM calls a process. The alternative — a value threaded through every module call — would have every module answer its advanced stream, so the coordinator's step would be rewritten for a number generator. The process keeps the modules pure and the ordering property exact.

What it costs, and what it does not. A draw is a call, and a step's draws come in the order the modules are stepped. That is why the coordinator now steps sensor modules one at a time: with a barrier, which module drew first would be decided by which call arrived first. Monty's own loop is one call after another for the same reason, so a step's numbers are unaffected — a step's wall clock is the sum of the sensor modules rather than the slowest.

Measured. Against Monty's own world (scripts/probe_world_episodes.exs, base_config_10distinctobj_dist_agent), with Monty.Frameworks.Experiments.Seed giving the episode seed Monty's own episode_seed column: the first action of the first episode is turn_left 16.782644145270698° + look_up 20.802956601860625° in both. Before the stream it was turn_left -33.06° + look_up 11.29° — the salience module's dither, drawn from a stream that was not Monty's, pushing LookAtGoal towards a different point of the same view.

What is still not Monty's, and why it is no longer this. Two episodes of ten agree with Monty step for step (21 and 28 steps), and eight do not: 21/27/28/36/40/34/38/37/46/45 here against 21/28/28/26/42/21/35/50/47/29. The stream is in step for the first seven of them, though — the same number of 32-bit words consumed at every step (228, 40, 400, 420, 512, 456, 6) — and the first action that differs is chosen while both streams stand at word 184. So the two runs draw the same numbers and decide differently: the difference is in goal selection at the step the learning module's evidence first drives the motor system, and only from step 8 do the draws differ as a consequence of the different view. That is a decision to compare, not a generator to port.

A camera's message noise draws from the stream too, since 2026-11-06. It used to draw from Monty.Context.rng_for/2 — an Erlang :exsss stream of the sensor's own, one per step — so a config with noise_params made none of Monty's numbers and spent none of the episode's stream: every draw after it belonged to a different experiment. DefaultMessageNoise now draws NumPy's normals from ctx.rng (Monty.Random.Stream.normals/4, and random/1 for a boolean's probability), which is where Monty's rng.normal(0, sigma, shape) and rng.random() read. The numbers themselves are pinned against numpy.random.RandomState(7) in camera_sm_test.exs.

Measured, unsupervised_inference_distinctobj_dist_agent, seed 42, episode 1, step 0 — the first observation of the episode, per graph's highest evidence, against a run of Monty's own captured from its dashboard stream (contrib/liveview_experiment broadcaster, zmq_port):

So the world, the positioning, the patch, the features and the feature evidence are Monty's to the last bit — the largest disagreement in the table is one unit in the last place. With the noise on, the same step reads 0.650 / 0.672 / 0.445 where Monty reads 0.856 / 0.880 / 0.710, and that is the remaining piece: a noise_params is a dict in Monty and a map here, so the features are drawn in the YAML's order there and in term order here, which assigns the same count of draws to different features and — because rk_gauss keeps one spare value — changes which random_sample calls interleave with it. Carrying the YAML's key order into the composed config is the work that would close it.

Draw sites still on a per-module stream: DepthTo3DLocations noise, Monty.SensorModules.SensorModule's transform pipeline, and ActionSampler.stream/3.

2.2 motor_only_step travels in the result

In one line: the flag rides in the result instead of being set on a shared object.

The difference. Monty hands the motor system to the selector (fixme_provide_motor_system) so a policy can set motor_only_step on it directly. Here the policy says it in the MotorPolicyResult and the motor system carries it.

The code. motor_policies.py:1120-1125 — the hack, in Monty's own words. Ours: motor_policy_result.ex and motor_system.ex.

Why it is deliberate. The same effect on the step, with no shared mutable object between policy, selector and motor system.

2.3 The motor system answers with itself

In one line: every call answers the next version of itself, because a struct cannot be changed in place.

The difference. MotorSystem.call/6 answers {actions, motor_system} rather than only the actions. Same for the selectors and the policies.

Why it is deliberate. The record of what was decided has to live somewhere, and a struct cannot be changed in place. Monty mutates itself; this returns the next version of itself.

Where. motor_system.ex, motor_policy_selectors.ex.

2.4 get_output/1 answers {message, lm}

In one line: {message, module}, so every step in the port has one shape to compose.

The difference. Monty's returns the Message alone; ours answers it with the module, like every other step in this port.

Why it is deliberate. One calling convention for every step, so composing steps does not need to remember which ones answer bare values.

What to do about it. A caller that wants the confidence reads elem(result, 0).confidence. See EvidenceGraphLM.get_output/1.

2.5 A sampler answers the stream it leaves behind

In one line: {action, rng} — the caller threads the stream, so several draws in one step get different answers.

The difference. Monty writes self.action_sampler.sample(self.agent_id, ctx.rng) and mutates ctx.rng in place, so one stream advances across the episode. Here ActionSampler.sample/3 answers {action, rng} and the caller threads it; a policy starts its stream from the context on the first draw and reset/1 forgets it.

The code. Monty: action_samplers.py:61-71. Ours: action_sampler.ex, used by base_policy.ex, informed_policy.ex and surface_policy.ex.

Why it is deliberate — and why it is not about speed. A policy draws more than once in a step: SurfacePolicy samples a tangential direction and a distance, and the search draws a random angle. A stream keyed to the step alone would answer all of those draws with the same action, which is wrong rather than slow. scripts/bench_samplers.exs records that threading is also the cheaper option (1.0 µs against 1.4 µs per draw, in a ~39 ms step), but that is a tie-breaker, not the reason.

What it costs. The same as §2.1: the numbers a draw produces are Monty's for a given stream, but the stream is not Monty's.

2.6 get_top_two_pose_hypotheses_for_graph_id/2 answers {best, nil}

In one line: "there is no second hypothesis" is a value here, not a raise.

The difference. Monty raises where this port returns a value — see §1.4.

Why it is deliberate. A goal generator running without a hypothesis to test is a normal state, not an exception, and a crash in the middle of an episode is worse than a goal that is not proposed.

Where. evidence_graph_lm.ex.

2.7 depth_at_center is a function, and the policies delegate to it

In one line: the centre-pixel read lives with the procedures; policies reach it through their own namespace.

The difference. Monty keeps it in positioning_procedures.py, as a @staticmethod of PositioningProcedure at line 75. Here it is PositioningProcedures.depth_at_center/3, reading the same shape — observations[agent_id][sensor_id]["depth"], centre pixel — and MotorPolicies.depth_at_center/3 delegates to it.

Why it is deliberate. A behaviour cannot carry a static function, so the question lives with the procedures and the policies reach it through their own namespace, which is where the policies' code already asked it from while the environment layer was unported. The call sites read as they did.

2.8 Small vocabulary differences

In one line: atoms for closed sets, tuples for quaternions, and Monty's units wherever a number could differ.

• PolicyStatus is :ready | :in_progress here, "ready" | "in_progress" upstream; the strings appear only on a wire. → policy_status.ex

• rotation_quat is a {w, x, y, z} tuple here and a quaternion object there; ActionJSONDecoder converts on the way in from JSON lists. → action_json_decoder.ex
• Monty's fixme_undo_last_action raises TypeError("Invalid action: …"); ours is undo_last_action/1 and raises ArgumentError naming the kind of action it was given. The fixme_ prefix is Monty saying it would rather have Action.undo, and the name here leaves the complaint out. → motor_policies.ex
• Actions keep Monty's units (degrees, metres, wxyz), deliberately: an action built here must be the same object an environment there would perform. → action.ex

Why it is deliberate. Atoms for closed sets, tuples for quaternions, and Monty's units throughout: the vocabulary stays Monty's where it would show up in a number, and becomes Elixir's where it would not.

2.9 Inheritance becomes public functions and a module-dispatched hook

In one line: a public function plus a module-dispatched hook replaces Python's class hierarchy — same actions, different address.

The difference. Monty's SurfacePolicy is an InformedPolicy: it inherits the jump machinery and overrides __call__, reset and _handle_successful_jump. Elixir structs do not inherit, so the port does two things Monty does not have to:

• the jump step is a public InformedPolicy.goal_driven_actions/4, taking any policy that carries the jump fields (a jump_state shape) and answering the policy it leaves behind — what SurfacePolicy calls instead of inheriting;
• Monty's _handle_successful_jump is handle_successful_jump/1, dispatched to the policy's own module by MotorPolicies.successful_jump/1, because a private function cannot be overridden by another module's struct.

The code. Monty: motor_policies.py:1077-1100 (class SurfacePolicy(InformedPolicy)). Ours: informed_policy.ex, motor_policies.ex, motor_policy.ex (the optional callback) and surface_policy.ex.

Why it is deliberate. Nothing observable changes: the same step produces the same actions and the same policy state. What changes is where the code lives, and that the shared shape is a type (InformedPolicy.jump_state/0) rather than a class hierarchy.

2.10 Missing coverage counts as zero

In one line: a coverage nobody asked for reads as zero, instead of raising KeyError.

What Monty does. A sensor module's percept is built from the middle pixel, and a surface sensor may still extract features with that pixel off the object as long as part of the patch is on it — so the condition reads the coverage it computed a few lines earlier:

        if on_object or (self._is_surface_sm and features["object_coverage"] > 0):

features["object_coverage"] only exists if the config asked for that feature, so a surface sensor that declared some other features and whose middle pixel is off the object raises KeyError (sensor_modules.py:222). Every config Monty ships that sets is_surface_sm: true also asks for object_coverage, so the crash is latent there.

What this port does. Reads a coverage that was not asked for as zero (observation_processor.ex), which answers the same question — "is any of this patch on the object?" — without making a missing feature a crash. The percept itself still carries only the features the config declared.

2.11 A sensor with no surface answers nil

In one line: a sensor that saw no surface says so; it does not invent a location no pixel supports.

What Monty does. CameraSM.step always answers a percept: the processor reads observation["semantic_3d"] and indexes its middle row, so an observation that never went through the depth-to-points transform raises KeyError rather than reporting nothing (sensor_modules.py:196). Monty's own environments always run the transform, so the path is never taken there.

What this port does. ObservationProcessor.process/2 answers {:error, :no_point_cloud} (observation_processor.ex) and CameraSM.step/4 answers nil (camera_sm.ex) — a sensor that saw no surface, which is a different thing from a sensor that saw a surface it could not fit. The runtime already has a shape for a sensor with nothing to say on a step: a module whose percept is nil contributes nothing to the learning modules' inputs.

Why it is deliberate. The stand-in world in this port can answer an observation that has not been through the transform — the view finder's observation, before a transform the port has not written runs — and a percept invented for it would carry a location that no pixel supports. A percept describing nothing is worse than no percept at all.

2.12 The sensor's pose is composed once

In one line: one function composes a sensor's world pose, called by both sensor modules — one rule, not two copies.

What Monty does. The sensor's place in the world is the agent's pose with the sensor's own offset turned and added, and its rotation the two rotations multiplied. Monty writes those two lines in two classes — CameraSM.update_state (sensor_modules.py:565-570) and SensorModule.update_state (sensor_module.py:193-198):

        self._sensor_state = SensorState(
            position=agent.position
            + qt.rotate_vectors(agent.rotation, sensor.position),
            rotation=agent.rotation * sensor.rotation,
        )

What this port does. One function, AgentState.sensor_pose/2 (agent_state.ex), which both sensor modules call. It also owns the error for an agent that has no such sensor, so the two classes cannot disagree about what that means.

Why it is deliberate. The two copies are one rule, not two: a change to how a sensor is placed — a different offset convention, a normalised rotation — would have to be made twice in Monty, and a run that got it once would disagree with itself. Nothing observable moves: the same numbers come out of both callers, and a test pins the composition through each of them.

2.13 Two protocols and a mutable transform become one behaviour and a threaded value

In one line: two Protocols become one behaviour, and a transform answers the transform it leaves behind instead of mutating itself.

What Monty does. tbp/monty/sensor_modules/sensor_module.py declares RuntimeSensorModule and Transform as Protocols, and tbp/monty/experiment/sensor_module.py declares ResettableSensorModule and ExperimentTransform. The protocols exist so a runtime can ask isinstance(module, RuntimeSensorModule) or isinstance(transform, ExperimentTransform) — structural typing, not a base class. And a transform is a mutable object: the sensor module calls it and the object keeps whatever it learned (sensor_module.py:180-186).

What this port does. Three things, all of them the same discipline the rest of the port follows:

• a protocol that only asks "does this object have these functions?" is a behaviour here, and the port has exactly one sensor behaviour — Monty.Frameworks.Models.AbstractMontyClasses.SensorModule. So RuntimeSensorModule and ResettableSensorModule do not become modules at all (they are the only two classes marked :excluded in the module map for this reason); a sensor module is something with update_state/2, step/4 and reset/1;
• Transform is a behaviour, Monty.SensorModules.SensorModule.Transform, and a transform that can also be reset declares the second one, Monty.Experiment.SensorModule.ExperimentTransform — the same two callbacks, which is what Monty's protocol pair means;
• because an Elixir struct cannot be mutated, a transform answers the transform it leaves behind: call(transform, ctx, payload) :: {payload, transform}, and the sensor module keeps that one for the next step. A reset then has something to reset, which is what ExperimentTransform is for.

Why it is deliberate. Without the threading, a stateful transform — a smoothing filter, a background model — would be frozen at construction and would silently behave as if it were stateless. It is the same answer the port already gives for samplers (§2.5) and for every step in the runtime: the result comes out with the state it left.

2.14 The positioning loop is bounded

In one line: bounded at 100 steps, with an error that says so, instead of a run that hangs.

What Monty does. ExperimentEnvironment.pre_episode builds each configured positioning procedure and steps the environment until the procedure answers terminated or truncated, with no bound on the number of steps (environment.py lines 240-267). A procedure that neither reaches the object nor gives up therefore hangs the experiment.

What this port does. Monty.Runtime.Positioning.prepare/5 stops after :max_steps (100 by default) and answers {:error, {:positioning_did_not_finish, steps}}.

Why. A hang inside a supervised job is a wedged run that nobody can see; an error is a run that says what went wrong. Monty's own procedures finish in a few dozen steps on the shipped configs — a studio run of base_config_10distinctobj_dist_agent takes 25 — so the bound only fires where Monty would have hung.

2.15 Two recording environments, and only one of them is Monty's

In one line: two interpreters on purpose, and every recording says which one made it.

What Monty does. environment_arm64.yml pins python=3.8, numpy<=1.23.5 (because numpy >= 1.24 drops np.long), quaternion=2023.0.3 (same reason), scikit-learn=1.3.2, pytorch=1.13.1, pyg=2.1.0, pytorch-scatter=2.0.9, pytorch-sparse=0.6.15 and habitat-sim=0.2.2.

What this port records with. Two interpreters, on purpose:

• Monty's own conda environment tbp.monty — Python 3.8.20, numpy 1.23.5, quaternion 2023.0.3, scipy 1.10.1, scikit-learn 1.3.2, torch 1.13.1, torch_geometric 2.1.0, torch_sparse 0.6.15, torch_scatter 2.0.9, omegaconf 2.3.0, habitat_sim 0.2.2 — which is the pin list above. Anything that needs Monty's learning modules or its simulator is recorded here.
• A throwaway Python 3.13 environment built by scripts/setup_monty_oracle.sh for the recorders that only need arithmetic: numpy 2.4.1, OpenCV 4.10.0, quaternion 2024.0.13, scipy 1.18.1, omegaconf 2.4.0, scikit-learn 1.9.1, and torch 2.15.0 with torch_geometric 2.8.1 when asked for. Used where the conda environment is absent, and never where a Monty answer is what is being recorded.

Why. The pinned environment exists here, so parity recordings are made in it; the pip environment removes the "install conda, then 3.8, then Habitat" cliff from any other machine, at the cost of recording a newer library's answers. The two are not interchangeable and a recording says which ran it.

What this does not license. OpenCV is 4.13.0 in the pinned environment and 4.10.0 in the pip one; the fixtures that compare against OpenCV (§1.12, §1.13, §1.15) were recorded under 4.10.0. Re-recording them in the pinned environment would be measuring 4.13.0, which is a different measurement, not a fresher one.

2.16 Distractors are refused, not placed

In one line: {:error, {:distractors_not_supported, n}} for any n > 0, because the stand-in world holds one object.

What Monty does. OneObjectPerEpisodeInterface adds the primary target, then draws num_distractors names from the source list — everything but the target — places them around it, and records for each whether it is the primary target.

What this port does. OneObjectPerEpisodeInterface.place_distractors/2 answers {:error, {:distractors_not_supported, n}} for any n > 0, and change_object_by_idx/3 passes that up. Everything above it is Monty's: which objects a distractor may be drawn from (distractor_candidates/2), the target's semantic id, and the attributes the scene is asked to add it with.

Why. The stand-in world holds one object. Placing "distractors" it cannot hold, or quietly running an experiment that asked for a cluttered scene and got an empty one, would make a number that means something different from Monty's. A refusal is a run that says which feature is missing.

2.17 A recognition policy is given what it reads, not the model

In one line: four readings, not the live model — so a decision can be recorded and replayed.

What Monty does. A policy is __call__(self, model: MontyBase, count): it is handed the model object and reads whatever it likes off it — model.is_done, model.is_exploring, model.matching_steps, and lm.recognition_status.conclusion for each learning module. It also logs and telemeters every terminal decision through logger.info and telemeter.info.

What this port does. Monty.Experiment.RecognitionStrategy declares a model view — a map with exactly those four readings, the modules' conclusions flattened to a list — and each policy is a {module, state} pair dispatched by RecognitionStrategy.call/3. Constructors answer {:ok, policy} or {:error, message}, carrying Monty's ValueError sentences.

Why. The decision is what has to be compared with Monty's, and a decision that can reach into live processes is a decision that cannot be recorded and replayed. scripts/record_monty_recognition_strategy.py asks Monty's real classes 448 questions over the same view; the port answers every one of them the same way.

What is therefore not compared. The log lines and telemetry events Monty emits when a policy fires. They are the experiment's reporting, not its arithmetic; when the episode loop is wired to these policies, what it reports is its own concern.

2.18 A memory arrives as an export, not as a pickle

In one line: .pt becomes JSONL once, off the BEAM; a run is handed a copy, because a run learns.

What Monty does. A pretrained learning module is a torch pickle (.pt) holding GraphObjectModels; MontyExperiment loads it into the learning modules it has just built (initialize_model in monty_experiment.py).

What this port does. Monty.Studio.Pretrained.Torch reads that pickle in Monty's own environment and writes the graphs out as JSONL — priv/monty/libraries/<model>.jsonl.gz, one object per line, positions and the four matched features per node, every number at full precision (these files were once rounded to six decimals, and validation.md records what that cost). Monty.Studio.Memory reads one back into a GraphMemory, and a run builds its learning module around it (EvidenceGraphLM.from_config/2 takes an optional :memory). The library's README records which release and which command produced each file.

Why. The port cannot read a torch pickle and should not learn how: what a model is, for matching, is a position per node and a row of features per node, and both are plain numbers. Exporting them once keeps torch out of the Elixir runtime entirely.

The consequence worth stating. Decoding is about six seconds and the result is about four megabytes, so the decoded memory is cached per library path in :persistent_term and every run is handed a copy (about twenty milliseconds). The copy is not an optimisation detail: a run learns, so two runs sharing one memory would learn into each other's, which Monty never does — it loads the pickle afresh for each experiment.

And the mode. ExperimentMode is an enum of the strings "eval" and "train" in Monty; here it is :eval and :train (ExperimentMode) with to_string/1 and from_string/1 for the places a config or a log needs the string.

2.19 "Back where it started" is an angle, not the same floats

In one line: compared as an angle under 0.001°, because the quaternion round-trip is 1.1e-16 away.

What Monty does. InformedPolicy._assert_undo_jump_was_successful compares the agent's pose after an undo jump with the pose it stored before the jump, with np.all(state[agent].rotation == self._pre_jump_state.rotation) — exact equality, on quaternions, and it raises when they differ.

What this port does. The position is compared exactly, as Monty's is; the orientation is compared as an angle, and anything under a thousandth of a degree counts as where it started (InformedPolicy). The raise now says how far the agent was turned, so the number is in the sentence rather than something to reproduce.

Why exact equality cannot hold here. Monty stores the quaternion the environment gave it and hands that same quaternion back, so the comparison is between one value and itself. An agent's rotation here is rebuilt from the world's answer by normalising the quaternion the world reports (Rotation.from_quaternion/1), so a pose that is put back exactly still comes back 1.1e-16 away — measured by Rotation.to_quaternion/1 followed by Rotation.from_quaternion/1. Monty's assertion is not too strict for Monty; it is too strict for a rotation that makes that trip, and it fired on base_10simobj_surf_agent after ten objects, ending a run that had done nothing wrong.

What it costs. Nothing that is measurable: a jump that misses by even a hundredth of a degree still raises, and the raise now carries the angle.

2.20 A supervised episode's answer is a call, and an impossible config is refused

In one line: six numbers through one call per module, and two refusals where Monty would raise mid-run.

What Monty does. MontySupervisedObjectPretrainingExperiment runs its training episodes as exploration and then reaches into every learning module to state the answer the experiment already knows:

lm.detected_object = target.object_name
lm.buffer.stats["possible_matches"] = [target.object_name]
lm.buffer.stats["detected_location_on_model"] = self.first_epoch_object_location[target.object_name]
lm.buffer.stats["detected_location_rel_body"] = target.position
lm.buffer.stats["detected_rotation"] = target.euler_rotation
lm.detected_rotation_r = Rotation.from_quat(target.quat_rotation).inv()
lm.buffer.stats["detected_scale"] = target.scale

and, for a module it does not supervise, clears those fields so that update_ltm_from_stm has no detected_object to write a graph under.

What this port does. The same six numbers, in the same order, through one function per module: GraphLM.supervise/3 for a module the experiment names and GraphLM.clear_hypotheses/1 for the others (graph_lm.ex). Monty.Studio.Run.supervise/1 reads the target off the world and calls the model once, which is where the loop stops being able to reach into a module: a module here is a process, not an attribute (§2.9 says the same about every other method the model calls).

Two things follow from that, and both are refusals rather than stand-ins:

• GraphLM.from_config/2 refuses the base class. graph_1lm.yaml names GraphLM itself; Monty builds it and raises NotImplementedError on the second step, inside a run. Here the setup is refused before it starts, with the sentence "a bare graph module has no matching scheme to narrow possible matches with" (§2.6 of the hand-over is why this is a finding and not a stand-in).
• A supervised experiment against the stand-in scene is refused. The answer it states is read off primary_target, and the synthetic sphere answers none, so Setup refuses the run (:no_target_to_supervise) rather than crashing at the end of an episode that had already run.

What this does not change. The location_on_model a module is given is the object's place the first time the experiment showed it, not this episode's — Monty's first_epoch_object_location, carried by the run. A later epoch that places the object elsewhere still extends the graph it built the first time.

One repair came with it. Monty's Predefined and RandomRotation samplers write quat_rotation beside rotation, the same wxyz quaternion under a second name, and the world's primary_target is a PrimaryTarget — an object_name, a semantic_id and every sampled parameter (experiment/primary_target.py) — so every sampled parameter travels with the answer. This port dropped euler_rotation and quat_rotation when it copied a sampled object into its scene, so a target here had no turn to state. Both samplers carry quat_rotation now, priv/monty_py/world.py keeps it, and supervise/3 raises with the object's name if a target still has none. The target's own key is object_name here too, which is what Monty renamed it to once the type had a field of that name.

2.21 An object-orientation channel's evidence is ported but not yet dispatched

Monty's PR #1266 splits the scored distances in two — euclidean_distances for a channel whose pose is an object orientation, custom_distances for one whose pose is a surface — and scores the orientation case with a new _object_pose_evidence, the angle between two rotation matrices off the trace of their relative rotation. Both pieces are here, with their tests translated (GraphMatchingUtils.euclidean_distances/2, HypothesesDisplacer.object_pose_evidence/2), and flatness/1 now names its 0.5 as @minimum_curvature.

What is not here is the dispatcher that would choose between them. Monty reads it from the channel's own pose type (isinstance), and every channel this port builds so far carries a SurfacePoseVectors pose: the sensor modules that send an OrientationMatrix belong to the experiment/ layer, which the module map still lists as planned. Wiring it before there is a channel to dispatch on would be a branch no run can take.

Checked and found consistent

Recorded so the next reader does not repeat the audit. The first two are claims Monty's documentation makes; the rest are claims the code makes about itself.

• docs/how-monty-works/motor-system.md: a selector chooses among policies, the selector is what goals reach, and policies are the only things that produce actions — the shape of MotorPolicySelectors and MotorSystem here.
• docs/how-monty-works/motor-system/policy.md: "if the sensor patch moves off the object, the previous action is reversed" is InformedPolicy's step exactly, and InformedPolicyRandomWalk's too — the distant_2 and distant_5 motor-system configs pass the latter as their selector's default. The alpha momentum that page also mentions is SurfacePolicy's tangential-direction weighting, not the random walk's: a random walk samples independently on every step.
• get_output() returning a Message makes parent_lm.get_output().confidence correct in the evidence goal generator (see §2.4 for our shape).
• threshold_possible_matches, x_percent_threshold and the x_percent_scale_factor application agree with their own comments.
• MotorSystem keeps each step's actions with the state they were chosen in, resets that record per episode, and keeps the selector's own record for state_dict, as its comments say.