# `Monty.Observations.SensorObservation`
[🔗](https://github.com/d-led/ex.tbp.monty/blob/v0.1.0/packages/ex_tbp_monty_core/lib/monty/observations/sensor_observation.ex#L1)

One view of the world, as a sensor module receives it: Monty's
`observations.py::SensorObservation`.

Monty spells the shape as a `TypedDict` with every field optional, because a
sensor sends what it can and a consumer reads the keys it understands. Here it
is a map with string keys — the same shape the depth-to-3D transform writes and
every consumer in this application reads — plus the names for the keys a
consumer cannot do without:

  * `"rgba"` — the patch as rows of `[r, g, b, a]`, `0..255`.
  * `"depth"` — one distance per pixel, in metres.
  * `"semantic"` — one object id per pixel, `0` for background.
  * `"semantic_3d"` — one row per pixel, `[x, y, z, semantic_id]`, which is
    what a sensor module actually reads: the depth image is gone by then.
  * `"sensor_frame_data"` — the same points in the sensor's own frame.
  * `"cam_to_world"` — the 4x4 transform from camera to world coordinates.
  * `"pixel_loc"` — where in the image the patch was taken, when it was cropped.
  * `"raw"` — the untouched image, for logging.

## Why the accessors raise

A missing `"semantic_3d"` is not an empty observation, it is an observation
that never went through the depth-to-3D transform — and a filter that silently
read `nil` there would report a surface at the origin. The accessors name the
key they need, so the failure says which transform did not run.

# `float_rows`

```elixir
@type float_rows() :: [[float()]]
```

# `pixel`

```elixir
@type pixel() :: [0..255]
```

# `point_row`

```elixir
@type point_row() :: [number()]
```

# `rgba_rows`

```elixir
@type rgba_rows() :: [[pixel()]]
```

# `t`

```elixir
@type t() :: %{optional(String.t()) => term()}
```

# `cam_to_world`

```elixir
@spec cam_to_world(t()) :: float_rows()
```

The transform from the camera's frame to the world's, as 4x4.

# `depth`

```elixir
@spec depth(t()) :: float_rows()
```

The depth image, one distance per pixel.

# `fetch!`

```elixir
@spec fetch!(t(), String.t()) :: term()
```

The value under one of the observation's keys, or a sentence naming it.

# `point_cloud`

```elixir
@spec point_cloud(t()) :: {:ok, [point_row()]} | {:error, :no_point_cloud}
```

The patch's points in world coordinates, one `[x, y, z, semantic_id]` each.

`{:error, :no_point_cloud}` when the observation never went through the
transform, which is Monty's own answer to the same question.

# `put`

```elixir
@spec put(t(), String.t(), term()) :: t()
```

The same observation with one key set, which is how a transform writes.

# `rgba`

```elixir
@spec rgba(t()) :: rgba_rows()
```

The patch, as rows of `[r, g, b, a]`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*