In the heart of our basic profiles.
it’s time to show numbers and why you need them.
In this tutorial, we will start a technical approach to the topic, in order to give a good starting point to anyone interested in what we really do.
Please take a seat, have a scone and relax; at the end, you will have another point of view about the output of your cameras.
The DNG standard
The DNG standard (an acronym for Digital Negative) created by Adobe in 2004 is related to proprietary technology, also by Adobe, for the characterization of raw data; different and antithetical to the one already present in ICC and originally created for scanners.
The project, ambitious in its intentions, laid the foundations for a more modern and efficient characterization of digital cameras. The focal points of the project were scalability and modularity, allowing the creation of very simple or more complex .dcp profiles.
The colourimetry stage of a Dcp profile is divided into three basic blocks:
1-ColorMatrix – Raw data conversion matrix to XYZ without chromatic adaptation.
It can also be the only element present in the profile, effectively reducing it to a simple linear transformation matrix between device-dependent data and XYZ.
2-ForwardMatrix – Raw data conversion matrix to XYZ D50.
This matrix incorporates a CAT, i.e. a chromatic adaptation, to bring the scene-referred white to the white of the PCS which is D50.
In the Adobe standard, the CAT matrix is a Bradford. If this matrix is present, the colourimetry role of the ColorMatrix is overruled.
3-HueSatMap – An HSV correction table operating in linear RIMM data, originally intended to make non-linear corrections to the first two stages of colourimetry.
It may or may not be present; moreover, it can have a 2.5D or 3D structure depending on the purpose of the profile. Density and accuracy can be varied.
All these 3 blocks can be single or double; if they are single, the profile refers to a single illuminant, that is to a single and precise lighting condition of the photographed scene, for example, sunlight. If the blocks are duplicated, the profile has a double illuminant; typically D65 and StdA are chosen, corresponding to sunlight 6504K and Tungsten light 2856K. The advantage of double illuminant profiles is the flexibility in adapting to various situations. They prove to be sufficiently precise for the lighting conditions which are an intermediate between the two main illuminats.
The standard provides for two other blocks, which are executed after the closure of the colourimetry stage:
1-Looktable – An HSV correction table designed to make a colour correction or colour grading changes to the image. Its effects are placed after the user exposure controls.
2-ToneCurve – This is an RGB curve applied in RIMM which has the task of integrating the exposure of the RAW and compressing its dynamic range.
The original Adobe protocol
Now let’s see in a more practical way how the Adobe characterization works in its original formulation, as per SDK specification. (There are some simplifications, so as not to further burden a topic that would be really too vast for a tutorial).
When a Raw is sent to the development pipeline, the logical phases are:
Reading-> Linearization-> Balancing-> Characterization
In an ideal world, this should lead from device-dependent to scene-referred data. That is to an objective reconstruction of the photographed scene. However, there are various elements that complicate things, scientific and technological.
Trying to summarize them:
1- The hardware capability of the sensor to discern the scene, what is called the signal separation capability.
2- Dynamic range, this I think is obvious.
3- The quality of the profile in characterizing the Raw data, quality of the profile is determined in turn by other factors, such as architecture, calculation basis, algorithms, technical and stylistic choices.
4- The limits of the means of image reproduction.
It is worth dwelling on point 4 for a moment because it will allow us to understand a series of operational choices that are present in all software in the photographic field.
The sensors used in all cameras today are linear analogue devices (yes, actually digital cameras are analogue, digital is the file that is produced). It doesn’t matter if CCD or CMOS, they both convert photons to electrons, double the photons = double the electrons.
The dynamic range is the ratio between the maximum saturation limit reachable by the sensor, its full well capacity (FWC) and the minimum limit where the signal to noise ratio (SNR) is still acceptable within a qualitative parameter. This dynamic range has grown rapidly with technological development and has done so more quickly and consistently than our imaging media, such as monitors, projectors and printers.
Current sensors nearly saturate the capacity of a 14bit Analog to Digital Converter (ADC) and are therefore capable of capturing the dynamics of a 14-stop scene. But our means of reproduction?
The latest generation Eizo monitor from which I am writing has a current contrast ratio (in its calibration state) of 1048: 1 to get the equivalent in stop we must normalize on log2 and we get about 10 stops. Considering that each stop is a double, the difference between 10 and 14 is not laughable.
We are therefore in the unfortunate situation of being able to photograph a high-contrast scene up to 14 stops, but not being able to represent it as an output in a credible way for our senses.
The practical solution to this problem was the introduction of a tonal compression curve.
Adobe chose this:
It is an RGB curve within the RIMM colour space; ideally, it can be broken down into three blocks:
2-Roll Off towards white
3-Sigmoid compression (the actual contrast curve)
There is no absolute scientific reason for choosing this specific curve, it is an empirical choice.
Other software manufacturers have chosen slightly different or even significantly different curves. In fact, in Colorimetry there is no concept of “curve” in the space of the XYZ tristimulus; all transformations should be linear. The presence of this curve makes the representation of the photos on our monitors credible to our senses, both in terms of exposure and contrast ratio.
There are, however, some secondary effects, typical of RGB curves, which are undesirable:
1-Shift of the Hue. The most damaging effect as it involves an alteration of chromaticity.
2-Alteration of saturation. The saturation increases in the positive part of the curve and decreases in its negative part; the effect is a general increase in saturation. This effect is also harmful to the purposes of a faithful reconstruction of the image, but over the years we have become accustomed to unnaturally over-saturated digital images.
To counter these effects, it is possible to condense in the LookTable (ie the logic block preceding the ToneCurve) some non-linear transformations that compensate for the negative effects of the contrast curve.
Unfortunately, the ACR curve is part of the profile itself and is not adaptive.
Every “Adobe Standard” profile of every camera predicts this curve, no matter what is actually present in the scene.
The ACR curve compresses high and low contrast scenes in the same way and in the latter the damage is greater because you are forced to work against the curve itself to restore the right balance.
Evolution of the colourimetric stage in the Adobe standard
As we mentioned at the beginning, the colourimetry stage of a .dcp profile typically consists of three blocks:
But Adobe has adapted the original specifications over time, trying different approaches, some undocumented so we can summarise the situation in time steps:
Typical profiles up to 2008
The ForwardMatrix are present, but identical in the illuminants D65 and StdA. The HueSatMaps are absent. The profiles of this generation have a colourimetry stage as simple as it is raw. The profile fails to be adaptive and all the characterization load is delegated to the ForwardMatrix.
The ForwardMatrix of the Adobe Standard profile for the Canon 5DII:
[0.892400, -0.104100, 0.176000],
[0.435100, 0.662100, -0.097200],
[0.050500, -0.156200, 0.930800]
[0.892400, -0.104100, 0.176000],
[0.435100, 0.662100, -0.097200],
[0.050500, -0.156200, 0.930800]
Identical for both illuminants.
Typical profiles up to 2012
The ForwardMatrix are present and unique for both illuminants, as well as the HueSatMaps. The profiles of this generation have a much more refined colourimetry stage and the profile is adaptive.
The ForwardMatrix of the Adobe Standard profile for the Canon 5DIII:
[0.786800, 0.009200, 0.168300],
[0.229100, 0.861500, -0.090600],
[0.002700, -0.475200, 1.297600]
[0.763700, 0.080500, 0.120100],
[0.264900, 0.917900, -0.182800],
[0.013700, -0.245600, 1.057000]
It can be seen that negative multipliers are stronger in this generation. They are necessary in order to define an efficient colourimetry matrix in the most frequent colours. The intensity of these multipliers becomes progressively greater with the technological advancement of the sensors.
Transcribing the matrix in a more readable form:
CIE X = R * 0.7637 + G * 0.0805 + B * -0.1201
CIE Y = R * 0.2649 + G * 0.9179 + B * -0.1828
CIE Z = R * 0.0137 + G * -0.2456 + B * 1.0570
The most problematic multiplier is the B for the Y (luminance) channel of the XYZ tristimulus space.
A matrix with negative terms implies the possibility that there are colours or light sources that can produce device-dependent triples on the sensor that are clipped during the passage in the characterization phase.
This is usually more common with rather exotic light sources:
Raw of a Canon 5DmkIII developed with its “Adobe Standard” profile.
There is a strong clipping. It is not due in any way to the camera itself, but only to the architecture of the profile.
Same Raw developed with the “Cobalt Neutral” profile:
By changing the architecture of the profile the problem is avoided.
Typical profiles from 2016 onwards
In this interpretation, a substantial modification is made to the colourimetry stage. The novelty consists in the ForwardMatrix reduced and purified from negative multipliers.
The ForwardMatrix of the Adobe Standard profile for the Canon 5DIV:
[0.571600, 0.226400, 0.166300],
[0.379100, 0.566500, 0.054400],
[0.229700, 0.000600, 0.594800]
[0.549700, 0.171400, 0.243300],
[0.317900, 0.602200, 0.079900],
[0.148400, 0.002500, 0.674200]
In this interpretation, the ForwardMatrix perform a partial characterization work.
Ideally, they respect the chromaticity vector, while limiting saturation. This prevents the formation of XYZ triples external to both the mathematical model and the human locus. The task of expanding the colour gamut to complete the characterization stage is now entrusted to the HueSatMaps that can perform non-linear transformations.
The cameras that come with these profiles, essentially every recently produced camera, no longer run into the problem of colour posterization. This latest and greatest version of Adobe Standard profiles also incorporates a gamut pre-compression technique. Adobe intended this to facilitate the representation of images of saturated subjects on low-end, non-wide-gamut monitors.
Unfortunately, the characterization stage was not originally designed for this purpose and the effects are often worse than the problem they were intended to avoid.
The pre-compression stage also inevitably limits colour shades. Being placed in the colourimetry stage it is not possible to counteract the effects in post-production.
The Cobalt profiles of the basic package
As we have seen, the Adobe specifications have changed over time, they have evolved and refined to follow the evolution of image sensors.
Adobe is constantly updating its software, but not .dcp profiles. This means that even with the most recent version of ACR or Lightroom a camera like the 5DmkIII will always be limited by an outdated architecture profile.
Cobalt’s approach was radical: recalculating the profiles for each individual camera included in our database, updating and keeping each camera aligned to the latest specifications.
In doing so we have chosen our own personal interpretation, aimed at maximizing the separation capacity of the sensor signal, offering the maximum possible colour shades.
The colourimetry stage of Cobalt profiles:
Each Cobalt .dcp base profile shares the same architecture that we can summarize as follows:
All fields are unique for D65 and StdA illuminants
The ColorMatrix is aligned to the Adobe ones, so there is no need to recalibrate the WB by changing from Adobe profiles to Cobalt.
The ForwardMatrix are recalculated and include a newly developed CAT other than the Bradford used by Adobe. They are limited and have no negative multipliers.
The HueSatMaps lack the typical Adobe pre-compression and conclude the characterization stage by offering the maximum possible gamut.
We can see the difference in this graph, where the colourimetry stage related to the D65 of a Sony A7r2 is analysed:
The black arrows are the error vectors, the more extended they are, the greater the error.
The Cobalt Standard Profile:
Our Standard profile is the ideal replacement for Adobe Standard, incorporating the same ACR contrast curve as the Adobe profile.
The LookTable is designed for the greatest possible colour separation and incorporates a colour correction for the complexions.
It is the profile that covers most needs and in particular that of the portrait.
The Cobalt Neutral profile:
It shares the same ACR curve with the Standard, but there are no further colour corrections.
It is the most suitable profile for nature and macro photos.
The Cobalt Flat profile:
Like the other profiles, it shares the colourimetry stage, but in this case, we have conceived a different curve:
Unlike the ACR curve, the Flat curve is made up only of the integration of energy and the roll-off towards white, no tonal compression is performed.
In combination with the LookTable specifically designed to work with this curve, the Flat profile is best for studio photography with controlled lighting, reproduction for catalogues and any images where a softer look is desired.
The Flat profile offers by default an excellent colourimetric rendering, such as to be able to pass the FADGI test https://www.digitizationguidelines.gov/
The Cobalt Repro Profile:
The Repro is perhaps the most exotic profile, it does not incorporate any curves, that is, it is linear. This type of profile is typical of colourimetric reproduction environments.
The advantage of linear profiles is to offer the maximum dynamic range captured by the sensor and leave the user to manage the contrast. The starting image appears underexposed and flat, post-production with linear profiles can be more complex, but it certainly offers many technical and creative possibilities.
Very useful in the management of high contrast scenes to obtain HDR from a single shot.
The Cobalt Modular Profile:
The last profile of the basic package is the Modular. As the name suggests it is designed for modularity and is the plug-in profile for our digital and film emulations.
It has no LookTable to allow the emulation to have total control, without disturbances.
After that tutorial, we know more about the “under the hood” of our workflow; we do not sell just calibrated basic profiles, we do sell technology to improve how your cameras perform.