Monty.
Copy Markdown
View Source
The colour and blur operations VOCUS2's attention model is built from.
Mirrors tbp/monty/frameworks/models/salience/strategies/vocus2/images.py,
which is four lines of arithmetic per function and a call into OpenCV for the
rest. The arithmetic ports exactly; the OpenCV calls had to be measured.
Colour spaces
The attention model does not work on RGB. It works on a colour space where brightness and the two colour-opponent axes are separate, because "this patch is brighter than its surroundings" and "this patch is redder than its surroundings" are different questions and the second is invisible in RGB. Two of Monty's three spellings are pure arithmetic:
rgb_to_opponent/1— the classic opponent space, shifted to[0, 1];rgb_to_opponent_codi/1— Klein and Frintrop's variant, where the axes are not shifted, so a neutral grey is zero rather than a half.
The third, rgb_to_lab/1, is CIE Lab and goes through OpenCV in Monty. See
its documentation for what "measured" means there.
Which images
A single-channel image is rows of numbers; a colour image is rows of
[r, g, b], with the values as they were read: 0..255. Blurring is for the
single-channel images the pyramids are built from.
Summary
Functions
Blurs a single-channel image with a Gaussian of sigma.
Resizes a single-channel image to {rows, columns}.
The image in CIE Lab, shifted and scaled to [0, 1].
The image in the opponent colour space, shifted and scaled to [0, 1].
The image in Klein and Frintrop's opponent space, as VOCUS2 implements it.
Types
Functions
Blurs a single-channel image with a Gaussian of sigma.
Monty computes the kernel size itself — round(2 * truncate * sigma + 1),
forced odd — and passes both it and sigma to OpenCV, so there is no
auto-computed sigma to reproduce. The border is BORDER_REPLICATE: the edge
pixel is repeated outward, which keeps a pyramid's coarse levels from darkening
at their edges.
The kernel is a normalised Gaussian sampled at integer offsets and applied in two passes, as OpenCV applies it.
@spec resize(plane(), {pos_integer(), pos_integer()}, :area | :cubic) :: plane()
Resizes a single-channel image to {rows, columns}.
Monty passes OpenCV's interpolation constants; here they are :area and
:cubic, the two the attention model uses. :area averages what each
destination pixel covers, which is the right way to shrink an image: no
detail is dropped, it is averaged in. :cubic interpolates a cubic through
the four nearest source pixels, which is the right way to enlarge one.
Measured, not recalled
Both were checked against OpenCV 4.10 over random shapes and agree to float32
precision. Cubic uses OpenCV's coefficient A = -0.75 and its coordinate
convention — the destination pixel's centre maps to (i + 0.5) * scale - 0.5
in the source — and both clamp outside the image rather than reading a border.
:area dips below a pixel step when the destination is larger than the
source: OpenCV was measured to answer something that is neither area nor
linear there, and Monty never asks — the pyramids only ever shrink by area and
enlarge by cubic. This port answers the ordinary area average in that case;
docs/monty-differences.md §1.13 records the difference.
@spec rgb_to_lab(colour_image()) :: colour_image()
The image in CIE Lab, shifted and scaled to [0, 1].
Monty calls cv2.cvtColor(image, COLOR_RGB2Lab) and divides by 255. OpenCV
answers CV_8U for an 8-bit input, so L arrives as 0..255 for 0..100
and a, b arrive shifted by 128 — which is what makes the division by 255
the right way back.
Measured, not recalled
OpenCV 4.10 converts 8-bit RGB to Lab through integer lookup tables; this port
computes the standard sRGB → XYZ → Lab chain in floating point. The two are
close but not identical: measured over all 16,777,216 colours, the worst
disagreement is 2.67 units of 255 — about one percent of the range, in the
dark, saturated corner — which is OpenCV's 8-bit quantisation showing, not a
different colour space. docs/monty-differences.md §1.12 records it.
@spec rgb_to_opponent(colour_image()) :: colour_image()
The image in the opponent colour space, shifted and scaled to [0, 1].
One channel of brightness and two of colour opponency: red against green, and
blue against the mean of red and green. The shifts by 255 and divisions by 2
are what put an opponent axis, which is signed, inside [0, 1].
@spec rgb_to_opponent_codi(colour_image()) :: colour_image()
The image in Klein and Frintrop's opponent space, as VOCUS2 implements it.
The same two colour axes as rgb_to_opponent/1 without the shift, so a
neutral grey sits at zero and the sign of an axis means something. Brightness
is still [0, 1].