Advice on docu workflow with SD tape footage, FCPX or not?

[Bill Davis]

[Michael Gissing] “8 bit processor heavy and not lossless so regardless of source, a poor choice for an edit codec in a large project. In this day and age of cheap drive storage hard to justify it as a choice for the doco workflow being proposed here.”

Look I get the general scope of this. That’s always been the safe and sane way to look at these things.

What I’m trying to determine is if it’s the technically accurate and mesurable way to look at them.

There’s no question that if you have a high rez master, digitizing it via any type of lossy codec is a bad idea.

What I want to know is if the conventional wisdom that suggests that when you start with a relatively poor resolution analog master, applying the same codec if it makes things AS MUCH worse as the drop from high rez to compressed? And how much worse do things get.

This should be measurable shouldn’t it? Shouldn’t we be able to take a frame from a VHS capture and run it into H-264 and look at the results pre and post encoding and see just how much things change?

And is the change clearly visible or is it just sorta visible?

If it confirms our existing perceptions, fine. But if there’s “less visible loss” because we can’t easily perceive the difference between relatively low rez original and an h-264 compressed digitization of that low rez original – I kinda want to know that.

Seems like something worthy to explore. That’s all.

Know someone who teaches video editing in elementary school, high school or college? Tell them to check out http://www.StartEditingNow.com – video editing curriculum complete with licensed practice content.

[Andrew Kimery]

[Bill Davis] “What I want to know is if the conventional wisdom that suggests that when you start with a relatively poor resolution analog master, applying the same codec if it makes things AS MUCH worse as the drop from high rez to compressed? And how much worse do things get. “

How much depends on the the footage. An INTV with a guy in his living room will most likely be less taxing from a compression standpoint than footage of a campfire (the constantly changing flames) or a red carpet event (flashbulbs going off). The more that changes between frames the harder it is for compression to keep from macro blocking. Footage of running water, leafy trees or tall grass in swaying in the breeze, color gradients (like a wall lit unevenly by a diffuse light source), etc.,. are also good things to look at if you are stress testing a codec.

On a related note, a long time ago I remember reading a very detailed, objective comparison between BetaSP and DV done by Adam Wilt and, long story short, DV had higher image quality than BetaSP (all other things being equal of course). One thing Adam noticed though is that when both formats reached their limits in terms of resolving image detail, the BSP might be perceived as looking better (in a subjective sense) because the fine detail starts going soft as opposed to the blockiness you see with digital compression.

So even when coming from relatively low-res analog sources you might have to aim higher than you think you need to during the digital conversion to avoid the digital artifacting which people find more bothersome than it’s analog equivalent. I always try to stay with an intra-frame (ProRes, Cineform, DNxHD, etc.,) since the compression happens on a frame-by-frame basis and is less likely to fall apart as opposed to an inter-frame codec where groups of 12 to 15 frames are all compressed together. I worked at a place once where, due to the extremely high volume of footage they created on a weekly basis, ProRes LT was the default capture codec (prior to that it was DVCPro HD) and our image quality always passed QC for broadcast. I’m not one to automatically suggest taking VHS to ProRes HQ because, at some point, all you are doing is pouring a 12oz can of Coke into a 2 liter bottle.

-Andrew

[Bill Davis]

[Andrew Kimery] “n a related note, a long time ago I remember reading a very detailed, objective comparison between BetaSP and DV done by Adam Wilt and, long story short, DV had higher image quality than BetaSP (all other things being equal of course). One thing Adam noticed though is that when both formats reached their limits in terms of resolving image detail, the BSP might be perceived as looking better (in a subjective sense) because the fine detail starts going soft as opposed to the blockiness you see with digital compression”

Adam is a friend and I reached out to him for his thoughts on this specific circumstance.

When he gets back to me, I’ll post his thoughts.

Know someone who teaches video editing in elementary school, high school or college? Tell them to check out http://www.StartEditingNow.com – video editing curriculum complete with licensed practice content.

[Andrew Kimery]

[Bill Davis] “Adam is a friend and I reached out to him for his thoughts on this specific circumstance.

When he gets back to me, I’ll post his thoughts.
“

Cool, I look forward to hearing his 2 cents.

[Craig Alan]

While his answer should be informative, I highly doubt he’d recommend not leaving it in its original codec. That was the beauty of the SD DV days – shoot and edit in the same codec. Compared to any HD codec it takes up very little space so not sure why you’d do this.

My other thought is that the original DV tapes make a great way to archive them. Just play them once in a while and keep them in a temperature controlled place. even make a copy if they are valuable.

All media creators need a better way to archive this stuff. Cheaper/faster/long shelf life.

How ironic that film has longer shelf life.

I also don’t get why SSDs haven’t become way cheaper. Cheaper than they were yes but any where near HHDs, no. Just looking at them it would seem after the initial R&D of their form that they should be cheaper – no moving parts.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.

[Michael Gissing]

[Bill Davis] “Seems like something worthy to explore. That’s all.”

Perfectly valid to question what can be seen and what can be measured. When you start grading and watching the result on a 65″ 4k monitor 2 metres away is when you notice. I am forever rolling edits a frame to avoid a tiny camera bump or blink when onlining. And when grading you start to notice when the banding starts or the colors go weird. The small monitoring on most edit setups simply don’t let you see but eventually on the big screen the differences matter.

The other issue is processor load with highly compressed codecs like H264 or H265. I have a hugely grunty graphics card that lets me grade with high quality debayering on 5k RED files but with H264 it is the main processor, not the graphics card that has to do the initial decoding.

I take the point that there will be diminished returns capturing VHS to uncompressed 4k, but the issue of drive space to processor load is an easy one to solve with codecs like ProRes and DNxHD being perfect to get the balance between drive space, system overhead and quality being totally nailed.

[Bill Davis]

Adam responded and his answer, as I suspected, was extremely interesting! I neglected to ask for permission to quote him here publicly, but remedied that in a follow-up. I expect to get word from him tomorrow and will post the relevant parts his reply as soon as I have permission. I’ll just say it seems my eyes were NOT fooling me. Stay tuned.

Know someone who teaches video editing in elementary school, high school or college? Tell them to check out http://www.StartEditingNow.com – video editing curriculum complete with licensed practice content.

[Bill Davis]

Just got permission to post my correspondence with Adam on this…

Sounds like once a signal is encoded to any of the relatively “low bandwidth” media like VHS, Hi-8 or even DVCAM, what’s lost is high frequency detail – and that’s not coming back.

So if you’re digitizing that kind of content – h-264 encoding isn’t going to damage the remaining signal very much at all.

Here’s the transcript of Adam’s responses to my questions (minus some irrelevant personal chatting on other topics) for those who want more details…
___________

Bill: Basically, I’ve been digitizing a lot of my old VHS, Hi-8 and DVCAM tapes through a BlackMagic Video Recorder dongle purely for long term archive convenience and have been surprised that the visual quality after digitization is significantly better than I expected. 

Which started me thinking whether if you START with such a low resolution master – does the TYPE of compression applied have as much effect on visible results as it does on better originals?

Adam: Not nearly so much. Low-band analog masters (and the chroma filtering in 4:1:1 DVCAM) have a lot less compression-unfriendly detail in them than a higher-resolution source, so the compression has an easier job of squishing it down: all the tricky bits have already been thrown away, more or less. You can use a higher compression ratio, or more aggressive compression overall, before you start to see a noticeable quality hit; the recording format has already “compressed” the source image to a certain degree.

Basically will the VISIBLE effect of compressing low rez originals to something like h-264 be as BIG a hit as applying similar compression to a better quality master?

Nope, the quality difference won’t be nearly as visible.

Have you ever tested anything like this? Your thoughts?

When DV25 came out I was stunned that I could edit to it from BetaSP or any of the color-under formats and not see ANY generation loss. The “prefiltering” applied, even by BetaSP, had already removed high-frequency detail and chroma that DV25 would have a hard time with.

Cheers,
Adam Wilt
______________
So there you go. If you’re transcoding low rez masters, don’t freak out if you can’t use a totally pristine dubbing chain. It might not get you much actual, visible benefits.

Know someone who teaches video editing in elementary school, high school or college? Tell them to check out http://www.StartEditingNow.com – video editing curriculum complete with licensed practice content.

[Michael Gissing]

[Bill Davis] “Adam: Not nearly so much. Low-band analog masters (and the chroma filtering in 4:1:1 DVCAM) have a lot less compression-unfriendly detail in them than a higher-resolution source, so the compression has an easier job of squishing it down: all the tricky bits have already been thrown away, more or less. You can use a higher compression ratio, or more aggressive compression overall, before you start to see a noticeable quality hit; the recording format has already “compressed” the source image to a certain degree.”

All interesting opinion although I don’t know Adam from, well, Adam. Using a long GOP encoding of DV material will of course further compress and DV is known for its blockiness so re-encoding such blocking is going to cause further degradation. But none of this addresses my other concern about having H264 as an edit codec in a long form doco.

I get that he is basically saying soft source without much chroma info isn’t hard to encode but it will lose quality and it isn’t a suitable edit post codec. Again my advice is ProRes or DNxHD. Life would be easier if H264 is kept out of the workflow until final masters are made for web or bluray.

[Andrew Kimery]

Thanks Bill.

Lower res formats inherently lacking fine detail (which is hard on both inter and intra frame compression schemes) makes sense. My concern was more about the changes between frames as opposed to the amount of detail in a single frame (so temporal vs spacial I guess). Things like camera flashes going off or a campfire fire dancing in the foreground are much tougher on inter-frame codecs than intra-frame codecs since the space-saving aspect of inter-frame codecs is based on the assumption that not much is changing on a frame-by-frame basis. I’ve seen red carpet footage from DV cameras that doesn’t show any artifacting beside clipping when the flashbulbs go off but red carpets shot with Sony’s EX1 (which uses the XDCAM EX codec) can turn into a pixelated mess for a frame or two if a flash lights up what the camera is pointed out.

Obviously the bit rate and the encoder have to be taken into consider as well as the more bits you have, and the better encoder you are using, the more robust the process is to handle rapidly changing images.

So, ultimately I guess my advice would be to prescreen footage and test sections that look like they might be problematic for H.264. Or if one has the storage space just capture everything in a codec like regular ProRes 422 or it’s DNx counterpart. It might just be paranoia, but I do share Michael Gissing’s concern of if you are essentially making new master copies of the footage why not err on the side of caution?

FWIW I grabbed some high res fire stock footage I have (1080p image sequence), turned it into DV using Adobe Media Encoder and then converted that DV file into H.264 using AME’s “Vimeo 1080p” preset and the results were surprisingly good. It was soft, obviously coming from DV, but it looked fine all things considered (no obvious or distractingly bad macro blocking from my quick look at it).