8 bit and 16 bit compositions..which to use?

[Andrew Somers]

Chris, first a week ago you used incorrect math related to pulldown rates while (wrongly) attempting to correct my pulldown rate list . Your post was deleted after I pointed out you were incorrectly rounding your answers. Now you are again using incorrect math and other misinformation to try and make some point but are again mistaken. I outline below:

Chris stated “Yes, fine, I knew dpx can be linear too.”

Do you mean video? Video is NOT LINEAR. Linear means gamma 1.0 – i.e. a straight line. Video has a gamma CURVE and is NOT linear. DPX/Cin is normally only used as a 10 bit container, and it is normally only used with a LOG curve or a VIDEO curve, but never (at least it should never) be used with a linear gamma.

https://prolost.com/blog/2009/9/30/passing-the-linear-torch.html

Christ stated: “DPX runs faster than EXR and EXR uncompressed runs the fastest out of all compressions for EXR. That is why DPX is still faster in nuke. “

This is not relevant to the discussion. Of course DPX is faster, it has a lower bit depth, but it also has SUBSTANTIALLY lower precision.

Chris stated: “You should only use EXR piz for archiving, ”

LOL. Oh really? Tell that to ILM.

Here is what ILM says in their spec for OpenEXR (edited here from brevity, see the full page at https://www.openexr.com/about.html):

History

ILM developed the OpenEXR format in response to the demand for higher color fidelity in the visual effects industry. When the project began in 2000, ILM evaluated existing file formats, but rejected them for various reasons:

• 8- and 10-bit formats lack the dynamic range necessary to store high-contrast images captured from HDR devices.
• 16-bit integer-based formats typically represent color component values from 0 (“black”) to 1 (“white”), but don’t account for over-range values (e.g., a chrome highlight) that can be captured by film negative or other HDR devices.
• Conversely, 32-bit floating-point TIFF is often overkill for visual effects work. 32-bit FP TIFF provides more than sufficient precision and dynamic range for VFX images, but it comes at the cost of storage, both on disk and in memory. W

Features

ILM decided to develop a new HDR file format with 16-bit floating-point color component values. …snip… For linear images, this format provides 1024 values per color component per f-stop, and 30 f-stops), with an additional 10 f-stops with reduced precision at the low end (denormals).

The half format supports denormalized numbers, positive and negative infinities, and NaNs. It is identical to the half data type in NVIDIA’s Cg graphics language, allowing a developer to process values from an OpenEXR image directly on current NVIDIA GPUs such as the GeForce FX family.

In addition to the half data type, OpenEXR supports 32-bit unsigned integer and 32-bit floating-point data types. OpenEXR images can have an arbitrary number of channels, each with a different data type.

EXR is an increasingly common intermediate format. EXR 16 bit float It is superior to DPX in terms of precision, and uses less storage (with lossless compression), and it is superior to 32 bit float TIF in terms of data storage.

EXR is especially useful in LINEAR colorspace workflows.

Again, let’s compare to DPX.

DPX/Cin is 10 bits per channel. Thus RGB takes up 30 bits. This is stored in 4 bytes, or 32 bit total, so 2 bits are discarded as waste. No data compression is used. The image data is either LOG or VIDEO gamma encoded. This allows DPX to be read very fast with minimal processing overhead. 22 years ago when Kodak began developing the Cineon System, hard drives cost upwards of $10,000 *per gigabyte*, and CPUs were very slow (Macs were still on 68K architecture, well before PowerPC even hit the market). Processing speed was a huge factor, along with disk space and bandwidth.

But that was decades ago. The current trend in this industry is toward higher bit depths in general.

10 bit has for a very long time been a common standard (SMPTE 259M, SMPTE 292M), used in HDCAM-SR and other formats.

Digital Cinema is 12 bit.

EXR is most commonly 16 bit float, though it can be 32 bit float, or 32 bit unsigned integer (and in fact, all three datatypes can be mixed in an EXR, as can image channels of multiple resolutions).

Now let’s compare:

DPX allows for 1024 values per color channel for the ENTIRE RANGE.

EXR 16 bit allows for 1024 values per color channel PER STOP, with > 30 stops range.

You shouldn’t need a calculator to see that EXR has substantially greater precision than DPX.

Chris Stated: “even if you don’t care about EXR’s mathimatically precisionless encoding. Can you say Disney archiving quality? nope!”

Again, a bizarre comment.

First, EXR can use 16 bit OR 32 bit float, and 32 bit unsigned integer. All have a *specific* precision, which I have already discussed at length in earlier posts on this topic. They are not “mathimatically precisionless encoding” and making such a statement demonstrates an ignorance of the topic.

SECOND, The Academy of Motion Picture Arts and Sciences is using EXR 16 bit float as the image container for the IIF/ACES archival and interchange standard.

https://www.digitalpreservation.gov/partners/documents/IIF_Overview_August_2010.pdf

https://www.stcatp.org/IIF_Background.shtml

Chris Wright stated: “DPX over-bright info is contained logarithmically in a 10 bit per channel file with 1024 levels of brightness per channel. It doesn’t get worse the brighter it gets. Where did you get that from?”

The LOG curve provides more data defining the darker image areas than the brighter ones because it is a linear representation of film density. A LOG curve is very much like a gamma curve. Put another way, if you add 1000 photons of light to a dark scene, you will increase code values faster than if you add 1000 photons of light in a bright scene. This is academic.

See these useful references:

https://www.acvl.org/digital_intermediates/dicompanion/ch03.html

https://acvl.org/digital_intermediates/dicompanion/apa.html#id597361

To make it more clear:

We’ll consider a LOG encoded DPX image with black (0%) mapped to value 95, and white (100%) mapped to 685 (the commonly accepted standard). (These relate to negative scanning densities where 95 is D-min and 685 is 90% white).

Light – DPX integer value (approx)

0% – 95
50% – 595
100% – 685
200% – 775
400% – 865
800% – 955
1600% – 1045 (over run, illegal value – max value is 1023, about 1350%)

DPX allows *less than 4 stops* over “100%” before running out of room.

Basically, assuming a negative gamma of 0.6, a DPX scan will increase by roughly 90 code values for each stop in exposure. This means that adding 0.1 to a level of 0.1, making 0.2, will be defined by an increase of about 90 (in DPX integer), but adding 0.1 to 0.4, making 0.5 will only increase code values by 28. Adding X amount of light to a dark scene will have a greater impact on light values (and thus code values) than adding the exact same X amount of light to a brighter scene.

Put another way, to get an increase of roughly 90 code values, you must DOUBLE the light in a scene. For an increase of 180 code values then you must QUADRUPLE the light in a scene. As a scene gets brighter you have to add more light to increase the same number of code values, so in brighter scenes, a given code value represents a larger amount of light.

Now, compare this to an EXR 16 bit float which features 1024 code values per stop.

DPX:90 code values per stop (assuming film gamma).

EXR:1024 code values per stop.

Which has more precision? EXR has more precision with 1024 values per stop than DPX with 90 values per stop, and EXR does not clip at 1350%. EXR actually supports infinite numbers (and NaNs).

Chris Wright stated: “And yes, you have to have a high end 10 bit graphics card with a 10bit pipeline. It takes more than a monitor to make it happen.”

NONSENSE.

There are a ton of ATI And nVidea cards that support 10 bit. I also have an NEC monitor with 10 bit support and a 14 bit internal LUT, as well as an HDMI 1.3 deep-color enabled projector.

I also have a $150 BlackMagic HDMI card that captures and outputs 10 bit uncompressed over HDMI.

Apple ProRes supports 10 bit natively, and ProRes 4444 supports 12 bit. Avid DNxHD also supports 10 bit.

HDCAM-SR is 10 bit, as are a ton of inexpensive cameras like the XDCAMs and even the new very-inexpensive Blackmagic Cinema Camera shoots in 12 bit RAW or 10 bit ProRes.

https://www.blackmagic-design.com/products/blackmagiccinemacamera/techspecs/

Chris said: “And as for conversion errors, I did a no colorspace, 32bpc render from integer to float, float to float. EXR fails in all types. Even uncompressed. All had roundtripping errors.”

If you selected no colorspace, then you can’t be in LINEAR space, now can you? In after effects, you *must* select a color space to enable LINEAR. Also If you are set to 32 bpc in AE, then you are in FLOAT not INTEGER. It does not do much good performing a series of tests that fail to use the software as intended.

You may find more answers in the following:

https://prolost.com/blog/2009/9/30/passing-the-linear-torch.html

https://vfxhaiku.com/2009/10/intro-to-32bpc-in-after-effects/