What a sensor saw, as a picture a page can show.
The world renders depth, colour and semantic id per pixel; this turns the first two into images. Monty's own dashboards do the same thing — the PyView prototype this studio follows puts a base64 PNG of the camera and one of the depth image side by side — and for good reason: an image is the only way to see a wrong percept rather than infer one.
Depth is drawn as greyscale over the frame's own range, and the range is written on the frame, because a depth image of a close surface is otherwise a flat grey square that says nothing. Pixels where nothing was seen (depth 0, which is what Monty's depth images use for "no return") are black, so an object against empty space reads as an object against empty space.
A frame's data is markup the page may interpolate as it stands. Everything in
it that is not a number of ours is escaped here, at the point the string is
built — a template that had to remember to escape it could forget.
Summary
Functions
The greyscale rows a depth image becomes, and the range they were drawn over.
The view from one sensor at one step.
The pictures a sensor's observation makes, base64 PNGs and the depth range.
Whether an observation has a picture in it.
Types
@type frame() :: %{ sensor_id: String.t(), step: non_neg_integer(), media_type: String.t(), data: Phoenix.HTML.Safe.t(), location: [float()], note: String.t() }
Functions
@spec depth_view([[number()]]) :: {[[non_neg_integer()]], {float(), float()} | nil}
The greyscale rows a depth image becomes, and the range they were drawn over.
Public because it is the picture's arithmetic rather than its presentation: a range that collapsed — everything seen at one depth — has to be visible as such.
@spec frame(String.t(), map(), non_neg_integer()) :: frame()
The view from one sensor at one step.
observation is the world's own: "depth" and "rgba" are images, one list per
per row. An observation that carries "images" instead has already been drawn —
a recorded step keeps the pictures rather than the four thousand numbers a step
they came from — and is shown as it was drawn.
@spec images(map()) :: %{ depth: String.t() | nil, colour: String.t() | nil, range: {float(), float()} | nil }
The pictures a sensor's observation makes, base64 PNGs and the depth range.
Answers %{depth: nil, colour: nil, range: nil} when the observation carries
nothing to draw — a world with no scene, or a recorded step from before runs
kept their pictures — because a frame that said nothing about what it was
looking at would be worse than one that says there is nothing.
Whether an observation has a picture in it.
What a replay asks before drawing a frame: a step whose sensor kept nothing drawable is not a step with two blank boxes in it.