Recap: XYZ Tristimulus Values

XYZ is the most important colour space in colour science. Colour appearance models, uniform colour spaces, and various RGB spaces are all derived from XYZ. XYZ values are obtained by multiplying spectral radiance by the colour matching functions, as follows:

$$ X = k \int_{\lambda} P(\lambda) \bar{x}(\lambda) d\lambda $$

Where \(P(\lambda)\) is the spectral radiance, \(\bar{x}(\lambda)\) is the standard observer’s colour matching function, and \(k\) is a normalisation coefficient.

Spectral Sensitivity Function

Most cameras use colour filters to split light into three channels. The transmittance of the filters and the wavelength response of the photodiodes jointly determine the sensitivity of the three channels to different wavelengths of light. This wavelength-dependent function is called the Spectral Sensitivity Function (SSF).

Following the calculation process for XYZ, the pixel value output by an ideal sensor can be calculated by the following formula:

$$ R = k \int_{\lambda} P(\lambda) \bar{r}(\lambda) d\lambda $$

Where \(\bar{r}(\lambda)\) is the spectral sensitivity function of the red channel, k is a coefficient, and \(P(\lambda)\) can be the spectral irradiance on the sensor.

The amount of light received by the sensor is determined not only by the light intensity but also by the camera’s shutter speed and the lens aperture size. For convenience, these are all represented by k.

Linear Transformation?

Although XYZ and RAW RGB are both linear spaces, this does not mean they can be directly transformed linearly by a single matrix. However, an approximate transformation is possible, which is the role of the CCM (Colour Correction Matrix) in an ISP.

From the formulae above, it is easy to see that the condition for being able to transform RAW RGB to XYZ using a 3x3 matrix is:

$$ \begin{aligned} \bar{x}(\lambda) = a_{11} \bar{r}(\lambda) + a_{12} \bar{g}(\lambda) + a_{13} \bar{b}(\lambda) \\ \bar{y}(\lambda) = a_{21} \bar{r}(\lambda) + a_{22} \bar{g}(\lambda) + a_{23} \bar{b}(\lambda) \\ \bar{z}(\lambda) = a_{31} \bar{r}(\lambda) + a_{32} \bar{g}(\lambda) + a_{33} \bar{b}(\lambda) \end{aligned} $$

If the above conditions cannot be met, the least squares method can be used to solve for an optimal CCM matrix.

Reference Implementation

Below is the SSF of a camera (from an early experiment, and should not be used for practical purposes), because I’m not sure how accurate the measurements are.

Spectral sensitivity function of a camera

Solving via the least squares method for the 3x3 matrix that linearly combines this SSF to approximate the CIE 1931 XYZ CMF yields:

$$ \begin{bmatrix} 1.297 & 0.0793 & 0.1033 \\ 0.558 & 0.569 & -0.1577 \\ 0.0596 & -0.1675 & 1.2465 \end{bmatrix} $$

Using this matrix to convert the SSF into an approximate XYZ (represented as Estimated XYZ):

Colour matching function derived from a linear combination of the spectral sensitivity function

Some Issues and To Be Continued

  1. The above content is based on assumptions such as: RAW is linear, and the camera’s spectral response range is 380-780nm.
  2. Methods for improving the CCM.
  3. Methods for creating a CCM without using an SSF.

Method for Evaluating CCM Quality

The purpose of a CCM is to convert RAW RGB to XYZ. A description I prefer is to ‘predict’ XYZ from RAW data. The essence of evaluating CCM quality is to see how accurate the prediction is. For XYZ, a colour difference formula can be used to calculate the difference between the predicted and true values. The colour samples used typically include neutral colours, saturated colours, and colours of common objects, such as skin tones, sky, and foliage. These ‘important’ colours are the colours found on colour charts. The most famous colour chart is the X-Rite ColorChecker Classic, which contains 24 colour patches, but X-Rite has not published its reflectance data, making it unsuitable for theoretical calculations. A better choice is the PMCC (Preferred Memory Colour Chart), which includes saturated colours, neutral colours, and 18 classic memory colours, and provides spectral reflectance data.

M. R. Luo, “The new preferred memory colour (PMC) chart,” Colour Research & Application, p. col.22940, May 2024, doi: 10.1002/col.22940.

Calculating the colour difference between the predicted XYZ and the true XYZ for these 30 colour samples allows for the evaluation of the CCM’s quality. By changing the light source (common choices include D65, A, and CWF or TL84), one can evaluate the CCM’s performance under different light sources.