# `Monty.Simulators.Synthetic.Scene`
[🔗](https://github.com/d-led/ex.tbp.monty/blob/v0.1.0/packages/ex_tbp_monty_core/lib/monty/simulators/synthetic/scene.ex#L1)

The one object the stand-in world has, and what a sensor sees of it.

A sphere, because a sphere is the simplest surface whose depth, normal and
curvature are all exact: the ray from a sensor either misses it, and the sensor
is looking at nothing, or hits it, and the distance, the outward normal and the
two principal curvatures follow in closed form. Anything a surface policy does
— following curvature, avoiding places it has been, re-orienting against a
normal — can be exercised against it, which is the whole point of having it.

The conventions are Monty's, from
`frameworks/utils/sensor_processing.principal_curvatures`:

  * a convex surface has **negative** principal curvatures (Monty's own TODO
    says it would rather they were positive; the sign is reproduced);
  * the curvatures are ordered largest first, which for a sphere is a tie;
  * `dir2` is the right-hand partner of `dir1` about the normal, so
    `dir1 × dir2 = normal`.

What it is not: an object. A run against this scene says nothing about
recognition; it says whether the body and the world agree about where the
agent is and what it is looking at.

# `observation`

```elixir
@type observation() :: %{
  on_object: boolean(),
  depth: float() | nil,
  rgba: [float()] | nil
}
```

What one ray saw: whether it hit the object, how far away it is, and the
colour at that point.

# `t`

```elixir
@type t() :: %Monty.Simulators.Synthetic.Scene{
  centre: Monty.Geometry.Rotation.vector(),
  radius: float(),
  rgb: [float()]
}
```

# `new`

```elixir
@spec new(keyword()) :: t()
```

A scene with `:radius` metres of sphere at `:centre`, coloured `:rgb`.

A centimetre-radius ball would be too small for a 2.5 cm standoff; Monty's
objects are of the order of a few centimetres, so five is a fair stand-in.
The colour is arbitrary — there is one object and it is all one colour — but
it exists because Monty's sensor modules report `hsv`, and experiments weight
it: a world with no colour would leave that feature empty.

# `observe`

```elixir
@spec observe(t(), %{
  position: Monty.Geometry.Rotation.vector(),
  rotation: Monty.Geometry.Rotation.t()
}) ::
  observation()
```

The middle pixel of a view, as a camera would answer it.

Answers the depth, colour and semantic id of one pixel — what
`Monty.Frameworks.EnvironmentUtils.Transforms.DepthTo3DLocations` unprojects,
and what a policy asking "how far away is the thing in front of me?" reads. The
middle pixel of a wide patch is not its axis — only a one-pixel camera or an odd
count puts a pixel there — so this renders rather than assuming.

# `radius`

```elixir
@spec radius() :: float()
```

The radius a scene has when nobody says: five centimetres.

# `render`

```elixir
@spec render(
  t(),
  %{
    position: Monty.Geometry.Rotation.vector(),
    rotation: Monty.Geometry.Rotation.t()
  },
  keyword()
) :: %{
  depth: [[float()]],
  rgba: [[[float()]]],
  semantic: [[float()]],
  centre: observation()
}
```

What a sensor at `pose` sees through a pinhole camera.

The camera is described the way Monty's own experiments describe it, so the
studio's world is the experiment's camera rather than one chosen here: a
`:resolution` of `{height, width}` pixels (a bare integer for a square patch),
a `:zoom`, and a field of view `:hfov` in degrees. Monty's Habitat agents give
a resolution and a zoom per sensor, and `DepthTo3DLocations` unprojects the
image with `fx = tan(hfov / 2) / zoom`, so a world that rendered a different
camera would make the transform disagree with it about which ray a pixel came
from.

A sensor looks along its own **−z**, as Monty's do, so its pose is enough: no
separate gaze has to be passed in and kept consistent with it. Depth is
measured along that axis, as a depth sensor measures it — not along the ray —
so an off-axis pixel's depth is the distance to the plane through the surface
point, which is what the transform assumes when it unprojects.

One pixel is the default, and it is a real camera rather than a summary:
depth, colour and semantic id come back as images, which is the shape Monty's
own code indexes. With one pixel the grid has a single sample at the axis, so
the field of view does not matter.

A ray that misses sees nothing: depth 0, which is what a depth image has where
the camera saw nothing, and a semantic id of 0. That is a difference from
Monty's worlds rather than a simplification — theirs are enclosed, so a camera
pointed anywhere sees a floor, a wall or a ceiling at some distance, and a
surface agent's search for a lost object relies on that
(`docs/monty-differences.md` §1.7). Here there is nothing behind the object.

---

*Consult [api-reference.md](api-reference.md) for complete listing*