Monty.Frameworks.Models.Salience.Strategies.Vocus2 (ExTbpMonty.Core v0.1.0)

Copy Markdown View Source

VOCUS2: where in a view is worth looking, as one map of a number per pixel.

Mirrors tbp/monty/frameworks/models/salience/strategies/vocus2/vocus2.py, and is the strategy a salience-configured experiment names. It builds a salience map out of four kinds of evidence and takes their weighted mean:

  • L, a, b — lightness and the two colour-opponent axes, each scored by ColorChannelSalience, so something is salient for being brighter, darker, redder or bluer than its surroundings;
  • depth — DepthSalience, so a nearer patch of surface stands out;
  • orientation — OrientationSalience, so an edge stands out whichever way it runs.

The map is then normalised, because a caller compares salience values against each other and an unnormalised scale would depend on how bright and busy the view happened to be.

The file's plain functions

Two combiners and two normalisers are plain functions in Monty rather than classes, so they live here instead of in modules of their own: map_max/1, map_sum/1, map_mean/1 combine named maps, and no_normalize/1 / range_normalize/1 normalise one.

Summary

Types

t()

Functions

call(strategy, ctx, rgba, depth)

The salience of a view: one number per pixel of depth.

from_config(config, options \\ [])

A strategy from a config, with its channels configured to agree with each other.

map_max(maps)

The strongest map at each pixel.

map_mean(maps)

The mean of the maps at each pixel.

map_sum(maps)

The sum of the maps at each pixel.

new(options)

A strategy from its parts.

no_normalize(salience_map)

The map as it is.

range_normalize(salience_map)

The map rescaled so its lowest value is 0 and its highest is 1.

Types

t()

@type t() :: %Monty.Frameworks.Models.Salience.Strategies.Vocus2{
  color:
    Monty.Frameworks.Models.Salience.Strategies.Vocus2.ColorChannelSalience.t(),
  color_space_converter: ([[[float()]]] -> [[[float()]]]),
  combine: Monty.Frameworks.Models.Salience.Strategies.Vocus2.MapCombine.t(),
  depth:
    Monty.Frameworks.Models.Salience.Strategies.Vocus2.DepthSalience.t() | nil,
  normalize: ([[float()]] -> [[float()]]),
  orientation:
    Monty.Frameworks.Models.Salience.Strategies.Vocus2.OrientationSalience.t()
    | nil
}

Functions

call(strategy, ctx, rgba, depth)

@spec call(t(), Monty.Context.t(), [[[float()]]], [[float()]]) :: [[float()]]

The salience of a view: one number per pixel of depth.

The colour image is converted to the strategy's colour space and its three channels scored one at a time; depth and orientation are scored too when the strategy has them; then everything is combined and normalised. Alpha is dropped, because Monty's camera writes a constant there.

from_config(config, options \\ [])

@spec from_config(
  Monty.Frameworks.Models.Salience.Strategies.Vocus2.Vocus2SalienceConfig.t(),
  keyword()
) :: t()

A strategy from a config, with its channels configured to agree with each other.

The depth channel takes the colour channel's sigmas, and the orientation channel a period of twice the centre sigma — both Monty's choices, and both of them choices a single config should not be able to make inconsistently.

map_max(maps)

@spec map_max(Monty.Frameworks.Models.Salience.Strategies.Vocus2.MapCombine.maps()) ::
  [[float()]]

The strongest map at each pixel.

map_mean(maps)

@spec map_mean(Monty.Frameworks.Models.Salience.Strategies.Vocus2.MapCombine.maps()) ::
  [[float()]]

The mean of the maps at each pixel.

map_sum(maps)

@spec map_sum(Monty.Frameworks.Models.Salience.Strategies.Vocus2.MapCombine.maps()) ::
  [[float()]]

The sum of the maps at each pixel.

new(options)

@spec new(keyword()) :: t()

A strategy from its parts.

The combiner defaults to a weighted mean of L, a, b and orientation at 1 and depth at 0.1. Depth is trusted less because a depth image's salience is dominated by the silhouette of whatever is in front, which is rarely where the interesting surface is.

no_normalize(salience_map)

@spec no_normalize([[float()]]) :: [[float()]]

The map as it is.

range_normalize(salience_map)

@spec range_normalize([[float()]]) :: [[float()]]

The map rescaled so its lowest value is 0 and its highest is 1.

A map that is already flat has no range to rescale and is clipped to [0, 1] instead, which keeps a constant map constant rather than dividing by zero. Monty asks np.isclose(range, 0), whose tolerance against zero is 1.0e-8.