-
More “demystification”…
I’m trying to stay out of this because in both software engineering and color theory – I’m pretty much an idiot.
I’m just an end user, and HAPPY to be one.
But this has blown up so much – that I’m fascinated.
I’m also IN LOVE with being in the world where I have access to ALL SORTS and LEVELS of expertise on complex topics that I don’t understand myself. Even if I don’t understand the math, at LEAST I can learn a bit from those that do.
In one of the X groups, Final Cut Pro X Editors, Phil Pan (the color science guy with what appear to be pretty deep credentials) has been weighing in on this OVERALL engineering debate.
Just to help keep things factual, here’s a chunk of his latest public posting there. Again, uncut and un-edited. Just a discussion point for those who are interested in the topic. If you want to engage directly or ask questions of Phil, this is happening in the Facebook Final Cut Pro Editors group.
With publicly graned permission to re-post and done verbatim with no alterations or commentary of any kind beyond some extra carriage returns I added to keep the WALL OF TEXT at a minimum.:
Phil Pan writes:
Maybe I should explain to users why Apple’s scope shows data excursion below “0” and above “100” IRE. In a previous comment I erroneously pointed that this puzzling, but this stems from the fact that I haven’t done video work in a long while (I mostly focus on film). If you come back to the standards litterature, this turns out to be entirely correct. It seems to throw people off and everyone thinks it’s a bug whereas in fact it’s the proper way to treat SDI signals.If you look at SMPTE 259 M and SMPTE 292 M, you’ll see that not all code values in an 8bit or 10bit range are used for pixel data. In an 8bit interface, for instance, codes 0 and 255 are used for sync, black starts at code 16 and white is pegged at code 235. The range of codes between 1 and 15 is called ‘footroom’ and the codes between 236 and 254 is called ‘headroom’. The range spanning 1 to 254 is called “Full Range”, and the range spanning 16 to 235 is called “Narrow Range” or more commonly “Legal Range”. The proper way to process pixel values in SDI video is to peg your computational “0” at incoming code value 16, and your computational “1” at incoming code value “235”. You perform your math as you should and then, because your algorithm sits out an array of numbers, you need to limit (‘clamp’) the minimum and maximum values to whatever your video standard will accept.
The recommended practice in SDI processing is to clamp not to the Narrow range, but to the Full range. It makes sense, because that’s precisely why there is a footroom and a headroom to start with. So this is what you’re seeing in Apple’s colour tools when you scope them. Negative Lift (aka “Shadows”) will cause negative excursions down to ~7%; positive Gain (aka “Highlights”) will cause excursions above 100 IRE, to ~109% and that is absolutely the official way to do this. Tools that clamp between 0 and 100 IRE are plainly wrong and Apple are showing they’re consciencious engineers. Why “-7” and “109”, you’ll ask ?
Here’s the math (for 10bit video; the ratios are the same as for 8bit): For footroom: (1–64)/(940-64) = -0.072. For headroom: (1023-64)/(940-64) =
What’s probably not helping users, however, is the nomenclature on the scopes themselves. Some scopes have an overlay that shows the extent of Full range vs Narrow range. If there was a coloured line at 109 IRE and one at -7 IRE, with a label that said “Full range”, maybe people would be less confused.
The habit, also, to state 0 to 100% as “IRE” units is antiquated but as we’ve seen with the whole metric-vs-imperial units, in America people like to stick to their trusted old values. Nothing (especially in a computing system_ would prevent a scope from displaying the vertical scale in code values. You’d clearly see that “0” is in fact code 16 in an 8bit workflow.
Lastly, it should be noted that the same thing applies to Rec 2020.
You need to understand that 709 and 2020 both use the *same* OETF function (BT 1886, which is a gamma of 2.4). Whether they’re in 8bit or 10bits, they have the *same* brightness dynamic range. What changes between 709 and 2020 is the chromaticities of the Red, Green and Blue primaries (the perceived colour of these primaries). 2020, because its primaries are more widely spaced, is able to represent more colours and is deemed “wide gamut”. This has nothing to do with HDR. In HDR, there is more appropriately Rec 2100, which defines both the PQ and HLG OETFs (so, a much wider brightness range), but still within the Rec 2020 primaries.
This is why in Final Cut, you have three options when working i WideGamut: 2020, 2020 PQ and 2020 HLG. The first one uses the same OETF as 709 (a simple gamma 2.4); the second uses the very broad PQ range which extends to 10K NITs, and the last one is the hybrid Log/Gamma curve that is basically the Rec709 OETF for dark tones, blending smoothly into an extened power function for highlights. HLG was designed for broadcast purposes, to maximise compatibility with non-HDR TV sets.
___
AGAIN the lines above are NOT mine – they are Phils. If you choose to quote the above text, please note it’s NOT material coming from ME. (the cow quoting system occasionally inadvertanly misattributes quotes of quotes) and this is NOT my writing. And needs to be attributed to it’s ACTUAL author. Thanks.
And thanks too to ALL that have participated in this – Especially Oliver and Simon – without their bringing up their original concerns – I would never have thought about any of this – and I feel better informed because of this whole discussion.
THIS is the new world of education. People sharing thoughts and ideas, more directly.
Thank you Engelbart, Kay, and countless, countless others for making all this possible.
Creator of XinTwo – https://www.xintwo.com
The shortest path to FCP X mastery. -
56 Replies
-
With respect to Philippe, this is indeed all valid but it is not actually illuminating of the phenomenon under consideration here.
I also notice that he seems to have dropped his “interesting” SMH theory of the color model being employed for the color wheels in 10.4.
Simon Ubsdell
tokyo productions
hawaiki -
Then Simon, please help me better understand this.
It appears (to my layman’s brain) that Phil (I don’t know him so I can only refer fairly to the handle he has elected to use in his posts) feels there are legitimate reasons Apple has made the operations interface choices it has. And that they are science based – not arbitrary or “bugs” in the sense they are mistakes that need correcting.
He appears to feel these are choices that users can easily adapt to in order to achieve the proper results they want consistent with accepted industry practices.
If you feel differently, by all means weigh in if you feel it’s worth your time.
The goal is enhanced understanding. For everyone. Nothing else.
My 2 cents.
Creator of XinTwo – https://www.xintwo.com
The shortest path to FCP X mastery. -
[Bill Davis] ” And that they are science based – not arbitrary or “bugs” in the sense they are mistakes that need correcting.”
I have said all along, despite all the talk of bugs, that this isn’t a bug, but nobody wanted to listen. (Just kidding, but seriously, it really, really, really isn’t a bug.)
It’s a decision. And there is an explanation for why it is happening. And Philippe’s observations do, in part, illuminate what is happening (but not I think in an especially helpful way).
But what we are seeing is nevertheless an anomaly. And an anomaly that is, in my view, not helpful to the actual user of the product, who after all is the most important factor in this equation.
I really don’t want to make this personal but I do need to stress that Philippe initially posted a remarkable comment on my YT channel to the effect that we were seeing the SMH color model in play in the color wheels, and I think I have conclusively shown that he was wrong about this. And of course he is now no longer espousing this view. We can all get things wrong and no blame attaches to him for having jumped to this conclusion.
The fact is that a midtones (gamma) correction absolutely needs to behave as if the zero and one values were the zero and one values that we see in the waveform monitor. This is a fundamental requirement to make grading adjustments manageable in the real world. (I tried not to complicate my account of this and consequently some might argue that I was guilty of an over-simplication, but for practical purposes, I was not.)
The Color Wheels do not conform to this requirement, as I hope I have conclusively shown.
The fact that they do not conform is most emphatically not an indication that Apple have adopted an exciting new color model that is of unique benefit to the user.
If you force my hand, I am going to have to say that what they have done is poor practice and very unhelpful to any and all users.
But again, it’s not a bug. So please, please, please stop using that word.
Simon Ubsdell
tokyo productions
hawaiki -
How scopes are shown/read should not have any effect on how the tools work. If one wants to see how the actual SDI signal looks one should check out a hardware rasterizer. Full/legal range and subblacks/subwhites have been a thing since forever and there have been workflows for those as well.
¯\_(ツ)_/¯
-
[Bill Davis] “He appears to feel these are choices that users can easily adapt to in order to achieve the proper results they want consistent with accepted industry practices. “
Phil is talking about how and why FCPX measures output on the scopes — not how or why the controls on the color corrector affect the image the way they do.
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
[Bill Davis] “Then Simon, please help me better understand this.”
I should probably reiterate what is at the heart of this problem and the reason Oliver and I thought it was worth investigating.
Whether you’re a professional colorist or just an everyday user of any color system, you absolutely do not want a midtones correction to dramatically affect your shadows and highlights.
The Color Wheels Midtones control operates much more like an Offset control than the Gamma control that it is unquestionably meant to be.
This is not helpful. And it is inefficient in practice.
I highlight the midtones correction because it throws into sharp focus an issue that ripples right across the entire implementation of the wheels.
Does that make the point clearer for you?
Simon Ubsdell
tokyo productions
hawaiki -
Just a stupid little question, but has Phil actually used or played with the color wheels using test signals or real world images? Or is his contribution purely theoretical? Fine either way, but it would help to know.
– Oliver
Oliver Peters – oliverpeters.com
-
[Simon Ubsdell] “Does that make the point clearer for you?”
Heck no – but that’s because I’m functionally a color idiot – and know far less about this stuff than either you or Oliver.
And obviously vastly less than Phil, too.
As an end user I have ONLY one question that’s of real use to me to answer.
Can I come to the tools (as implemented in X 10.4) and achieve the corrections I might require efficiently and accurately and in ways that don’t get me in trouble later?
Whether the METHOD of how the tools apply any math between the interface and the results is apparently an area of debate just like whether an editor prefer a mouse or a trackpad.
And that’s fine.
But can this tool GET THE JOB DONE is what has basically been called into question in this debate
10.4 was implied to be not just different – but WRONG.
It needed to be FIXED – STAT.
That was what I heard coming out of this discussion.ls early days
Now, apparently it appears to need to be “fixed” merely so it more closely meets the prior preferences of a certain class of colorists that might prefer that it work more like what they are accustomed to.
And don’t get me wrong – that quest is fair and legit.
But it’s “I need the mouse, not the trackpad” to be efficient. A defensible position for any individual editor. but not one fair to apply to ALL editors.
Not shipping a desktop machine without a mouse might be viewed as “non-standard” at one point in computing. Then Laptops become popular and shipping THEM mouselrss becomes just fine.
So is it “broken” (like you can’t move the cursor until this gets fixed – broken.)
Or just “different” like we’re giving you a trackpad not a mouse because we think this makes more sense here.That’s all I’m trying to figure out.
Creator of XinTwo – https://www.xintwo.com
The shortest path to FCP X mastery. -
The Facebook group is open to all.
10,000 plus members there, and he’s been happy to engage with anyone about this stuff, openly.
That he’s not here, is a shame. But probably an artifact of overall popularity of locations for information debate and discussion in the FCP X community.
This was once one of the MOST popular hubs of that.
Realistically, sadly, (honestly, crushingly to me personally) not so much anymore.
Sigh.
Creator of XinTwo – https://www.xintwo.com
The shortest path to FCP X mastery. -
[Bill Davis] “10.4 was implied to be not just different – but WRONG. It needed to be FIXED – STAT. ”
The comparison to alternative hardware like a mouse or trackpad is not relevant. This is not a viable alternative situation as the software is behaving differently depending on color space, in other words an internal inconsistency. Such inconsistency also creates a less efficient function using gamma adjustment in rec 709 color space. When a function correctly behaves as gamma in rec 2020 but not in rec 709, then Apple have to decide if they fix this or leave the inconsistency. Tip toeing around the problem, avoiding calling it a bug or whatever, the simple fact is the function is inconsistent and also inefficient. If I was beta testing this I would file a bug report and expect it fixed. I get that software people hate the word bug but to me an inconsistency within software would be filed as a bug report. To be kind I will call it a mistake. It is not a ‘different’ way of doing things as that would require internal consistency which is lacking in this case.
Apple have two choices. To fix it (and yes stat is important because it could lead to grading work done pre fix to not track and work having to be redone) or to ignore it which means they expect the user to adjust their grade practice when switching between color spaces and to also accept a behaviour in rec709 that is both different and requires more level adjustment which is inefficient.
Reply to this Discussion! Login or Sign Up