How Vegas handles Chroma Subsampling vs After Effects

[Cosmin Gurau]

Did you even read what I wrote? It has NO BEARING. It’s a grading difference, I tried this with over 10 clips, even ones with no grading, in which the images were identical, except for the blocky-ness.

[Norman Black]

I agree with this. It is well known that Canon and Nikon DSLRs output full range video, with the AVC VUI full range flag set to Yes. It is also known and Adobe normalizes these to studio video levels on import. Vegas does not. The stretch/compress of levels and cause an issue when your color is at the boundaries, and red can be problem color.

I can see a levels difference between the two photos.

Actually I believe the Nikon generates AVC Quicktime MOV files and not AVCHD format files. A technicality, but worth mentioning. For DSLR QT MOV files Vegas bypasses QT and uses its own AVC decoder.

[Cosmin Gurau]

I still don’t think it’s got anything to do with levels. It’s about how each softwares’ decoder works with the chroma subsampling. Here’s an extreme example I just recorded. No grading this time, just imported the video file and saved the first frame, from each software.

Sony Vegas:

After Effects:

This is unacceptable. The fact that, on no render setting can I obtain a high quality image from a proffessional software is just insane. Maybe there’s a workaround? Something I haven’t tried? Any sollution?

[Cosmin Gurau]

Sorin, my probably Romanian friend, you are showing me export render settings. That is useless, sorry. I can render to Lagarith (mathematically lossless) and still get the same problem, because the maximum quality image that vegas produces, that will be rendered into any choice of format – is the problem. If the horrible red channel blocky-ness is there, then it will be burned in the export, no matter what setting I use.

I’m looking for a way to get Vegas to DECODE my h.264 footage as well as After Effects does. That’s where the problem is.

[Cosmin Gurau]

This is the footage from my Nikon.

General
Complete name : C:\Users\Cosmin\Desktop\DSC_0865.MOV
Format : MPEG-4
Format profile : QuickTime
Codec ID : qt
File size : 17.0 MiB
Duration : 3s 378ms
Overall bit rate : 42.3 Mbps
Encoded date : UTC 2014-10-23 21:07:21
Tagged date : UTC 2014-10-24 00:15:25
NCDT : NCTG

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3s 378ms
Bit rate : 40.5 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.814
Stream size : 16.3 MiB (96%)
Language : English
Encoded date : UTC 2014-10-23 21:07:21
Tagged date : UTC 2014-10-23 21:07:21
Transfer characteristics : BT.470 System M
Matrix coefficients : BT.601

I don’t really understand what you mean, but I must mention that any Adobe application opens the file and displays it beautifully, like in the examples provided. After Effects, Premiere, Media Encoder. In fact, even MPC-HC, with its Enhanced video Renderer is almost just as good. VLC too, but not on DirectX, on OpenGL.

Quicktime player, however, displays it with the same blockiness. And from what I saw, Vegas uses Quicktime, whether it’s compoundplug.dll or qt7plug.dll.

Also, it’s probably worth mentioning this, from hurlbutvisuals.com blog:

Yes, H.264 is definitely not considered a finishing codec, but to be clear, Premiere Pro does not use it in that way. The H.264 is read natively by Premiere and once it is decoded into the app. it “resides” internally in a 32 bit float extended color space that is unmatched for color fidelity and dynamic range. Your tests at Laser Pacific have proven that. There is no need to transcode to Pro Res, although if one prefers to work with that type of intermediate, the Adobe workflow can handle it just fine…ProRes is a great intermediate codec, but depending on the original content, some information might be lost compared to how Premiere Pro decodes the same file. For a full explanation, please refer the poster to my longer explanation below:
Adobe CS5 reads the H.264 files natively into Premiere Pro and After Effects at the highest possible quality. Our color gamut and dynamic range for tonal detail from shadow to highlight is unsurpassed. There is even support for over-brights beyond 100% in After Effects. i.e. in plain English, we squeeze more out of these files than anything else out there! Shane Hurlbut’s filmout tests at Laser Pacific have verified that our interpretation of the H.264 is the smoothest and most filmic representation available. The magic comes from the use of proprietary interpretation algorithms and I might also mention that we bypass QuickTime for this process, which avoids the whole gamma conundrum. Once the file is living inside our apps on the timeline or project, we deal with the image information at the 32 bit float level. Now that is not saying we can make an 8 bit H.264 DSLR video capture look like perfectly shot IMAX footage scanned at 16 bits, but what we do offer up is the ability to edit, apply effects and color corrections within our apps. at an unprecedented level of quality.

So yeah, maybe that’s the reason Adobe’s decoding is far superior. Still, if VLC or MPC-HC can do a better job (not as good as Adobe, but still) than a professional, time-tested editing suite like Vegas, we are dealing with a rather SAD issue here.

Again, maybe there is a solution. Other than transcoding all my footage through Media Encoder to Lagarith, because I just don’t have the time and the space.

I’m still hoping. Anyone?

[Cosmin Gurau]

I did not assume it’s 4:2:2. I KNEW it was 4:2:0.

I wrote “it’s AS IF it’s a different camera. With 4:2:2. But I assure you, it’s not.”

Please, Sorin, READ what I write before replying. Please.

In the meantime, ANYONE? Any ideas on how to replicate this behavior in Vegas? This “chroma dithering”, if that is indeed what Adobe/MPC-HD/VLC are doing, how can it be achieved in Vegas?

[Cosmin Gurau]

I’m starting to think that these forums are pretty useless… It’s my third post and I have yet to actually receive any real help…

[Mike Kujbida]

Have you tried setting your Vegas Project Properties to 32 bit mode (found under the Video tab)?
I’ve never used this mode so I can’t comment on the Pixel Format settings (32-bit floating point (video levels) or 32-bit floating point (full range)) or the Compositing gamma mode settings (1.000 (Linear) or 2.222 (Video) to be used.
The following thread may help to clarify things.
What is the difference between 32-bit (video levels) and 32-bit (full range)?

[Cosmin Gurau]

I have. Zero benefits in this situation…