Can scrubbing avchd (h.264) footage be better?

[Bob Cole]

[Gerard Tay] “What the test shows is that if PPro is indeed using preview files on a same as source export, it does not decode and encode the preview file to the final format, otherwise the additional generation loss will show up in the test against the clip that was exported without using preview files. “

Thank you. This looks like valuable research. The only problem I have is that as a prospective PPro user, I found myself getting a bit confused. I’m sure it would be much clearer to me if I were a daily PPro editor, but right now I’m just lurking here, trying to get ready for the switch.

What is your bottom line recommendation as a “best practice” for exporting a top-quality render from PPro?

Bob C

[Tom Daigon]

[Gerard Tay] ” Thanks for the reply. Not to side track the topic, but anyway, I think the idea is that PPro decodes the preview files and re-encodes it in the output format. However, it seems that if the export setting is the same, the decode and encode does not happen.”

Gerard the process you are describing only happens in Smart Rendering, if we are talking about CS6.

https://blogs.adobe.com/kevinmonahan/2012/10/11/smart-rendering-in-premiere-pro-cs6-6-0-1-and-later/

Otherwise, the frames are decoded and recoded according to Adobe staff.

Tom Daigon
PrP / After Effects Editor
http://www.hdshotsandcuts.com
https://www.youtube.com/watch?v=DxPrG3WUyz8
(Best viewed at 1080P and full screen)
HP Z820 Dual 2687
64GB ram
Dulce DQg2 16TB raid

[Gerard Tay]

Thanks Tom. You can try this out yourself, actually. Pop a ProRes clip into After Effects, export the same ProRes clip from Premiere (to re-compress it, or you can do it with just about any software including FCP7 or Compressor), and bring that into After Effects. Then pre-comp it, set the top layer to blend mode “difference”, and drop a levels effect with gamma cranked up to max (the levels effect is 32bpc). Any generation loss from ProRes will show up on the matte.

A ProRes export isn’t a smart render (aka “use sequence settings” in FCP, “Same as source” in Avid), so there is a generation loss if the clip has no effects applied to it. However, if a clip has an effect applied and it is rendered (eg. you have broadcast safe or a color correction filter on the clip), when you export that part of the clip, and “use preview files” is checked, the exported file is exactly the same as if you exported it without rendering or if “use preview files” was unchecked. So either the “use preview files” option does nothing, or it copies the frames from the rendered preview files. But one thing PPro does not do is render off the preview files if the export format is the same as the preview format. At least not with a Quicktime Export and a create preview files to ProRes HQ at max bit depth.

[Gerard Tay]

The difference between this and smart rendering is that for smart rendering, if the clip is the same and no effects are added, the clip will not be re-compressed on output. But for clips with effects added, and if you’re exporting to the same format as your preview format, there is no additional generation loss if “use preview files” is checked.

[Gerard Tay]

Let me rephrase this answer.

The equivalent of an Avid “same as source” export and FCP7’s export “current setting”, is PPro’s “smart render”. But smart renders work for only for very specific formats.

There is the “use preview files” option, which is not the same as “smart render”. “Use Preview files” will use preview files to encode, and according to a differential matte test in AE, use preview files will copy preview files if the export format is the same as the preview files, skipping a double compression on those files, but clips that do not have effects applied will be rendered even if it is the same as the export format. So PPro, although not too intelligent, seems to be a tad smarter than we give it credit for. And you can do your own tests to verify this.

[John-michael Seng-wheeler]

[Gerard Tay] “But for clips with effects added, and if you’re exporting to the same format as your preview format, there is no additional generation loss if “use preview files” is checked.”

I think we need to dig into this further:

Adobe them selves have appeared to state that there is a generation loss when “use previews” is selected.

Your test was done with ProRes, which is a special case since Premiere can’t encode or decode the file directly. Quicktime does the decode, and passes the frame to Premere. The reverse occurs on encode.

I can’t figure out how smart rending could occur in this situation unless Quicktime is doing it for premiere.

JM

[Gerard Tay]

I repeated the test on another Prores HQ clip and I got the same result. There seems to be some form of smart rendering for clips with effects.

Now, when I export the clips as standard ProRes clips (SQ), and the previews were set to ProRes HQ, I get a generation loss when I check “use previews”. This would be expected, as AME is using the preview files for rendering/encoding and because the exporting format is different, the source clip would effectively have been rendered to ProRes HQ in PPro, then encoded to ProRes Standard (SQ).

The benefit of this behaviour is that if you have some processor intensive formats (eg. R3D), and because you may have rendered these clips during the edit, using preview files on export will save render time if AME didn’t have to go back and decode from the RAW R3D files, render the effect and then encode to H.264. Rendering off a lossy intermediate format (albeit perceptually lossless) would result in a slight drop in quality. So I guess this is what Adobe means.

I did a 3rd test by setting the preview format to ProRes SQ and exported to ProRes SQ, I ended up with the same result as the first test, which indicates smart rendering on clips with filters. Very smart, Premiere! Now if only it will do a “same as source” export like Avid/FCP7 or what PPro calls true “smart rendering”.

My settings so far is maximum bit depth and maximum quality renders, same for output. My source clips are ProRes HQ 720p25.

[Gerard Tay]

[John-Michael Seng-Wheeler] “Your test was done with ProRes, which is a special case since Premiere can’t encode or decode the file directly. Quicktime does the decode, and passes the frame to Premere. “

I’m not sure if you noticed, but ProRes behaves differently from a bunch of other clips that require a QT decode. Eg. DNxHD in a QT wrapper. QT decodes from Y’CbCr formats have notoriously suffered from the infamous QT gamma shift, as well as prematurely clipped highlights/sub blacks. DNxHD in QT does not seem to preserve Y’CbCr superwhites whereas ProRes does, so in a way ProRes functions similar to the uncompressed QT codecs which are open source. The same gamma shifting issue has plagued Avid editors who were trying to import ProRes into MC in the pre MC6 days before ProRes was “officially supported” by Avid. I suspect Adobe may have a different decode path for ProRes other than asking for a QT RGB decode. But this is pure speculation on my part.

[Reuben Fink]

I’m interested in this thread since I’ve been working with PP more over FCP recently. As far as I can tell you can’t scrub the play head manually with avchd footage as fast as prores. Currently I’m working on a 2008 Mac pro, 20gig of ram, GTX 570 2.5gig card and the footage is coming off of an internal stripe 0 raid configuration. I’m getting 350mb a sec. So definitely faster than your MBP but I am having similar problems. Although different from your experience the system can keep up when I hit the L key 4 times maxing it out. So I can either attribute it to the Raid speeds or the GTX Card or the combination. I’ll have to test further. What’s interesting though, is when I hit the J key while the footage is playing forward the system chokes. Also after hitting the L key if I hit K then J in quick succession it takes about 2 seconds for the footage to catch up and play backwards. Where it really falls appart is when I need to scrub with the mouse quickly. This has posed a big frustration as you have mentioned. I’m sorry but hitting the L key to it’s max is just not fast enough sometimes when you’re scanning hours of scratch footage and are on a very tight deadline. I’m currently doing a feature film EPK edit and it’s been a real pain to track footage down on a scratch timeline as quickly as I am used to in FCP. I’ve also tested this out on a much more powerful Windows machine with quadro cards and I’m seeing the same problem. But note this is not a PP vs FCP issue. It’s a highly compressed vs not so compressed issue. I rendered the same footage out as ProRes and it’s lightning fast in premiere pro. So yes I’m a little disappointed that I can’t edit AVCHD as fast as I had hoped but on the flipside I’m able to cut footage immediately when I have a director showing up in 3 hours and need to show him some dailies. This is I think is invaluable. And probably a trade off. I would like to know what configuration anyone has come up with that does work with AVCHD compressed footage as fast as ProRes if there is one. Maybe it’s striped SSDs? Or Dual SLI Cuda cards?

[Trevor Ward]

Bob and Tom, I wanted to circle back around on this issue. I had been using a new MBP 13″ (not the retina model). It has a 2.5 i5 with 16 GB RAM. I upped the RAM from 4 to 16 based on recommendations in this thread. Didn’t really notice a difference. I had been planning on sending you each a clip to test, but it got lost in the list of low priority things to do. Then a week later, my hard drive crashed. I got it fixed. then it crashed again a week later. This time, the Apple store wanted to do some extra stuff to it. I told them I couldn’t be without a computer for that long.

So I bought a 15″ MBP retina. Can’t remember the specs, but it was fast, though only 8GB RAM. Cost me $3000. I opened up the same project and this time, wow, big difference. Could scrub through my AVCHD footage almost like it was prores. Still bogged down a bit on some random AVC mp4 files (not even sure what they are).

After 2 weeks on the machine, I had to give it back. Apple “fixed” my 13″ and I opened the project today and got this message in a small window:

“This project was last used with Mercury Playback Engine GPU Acceleration (OpenCL), which is not available on this system. Mercury Playback Engine Software only will be used. ”

So there you have it. I think the reason my footage scrubbed poorly is due to the graphics card in this 13″ MBP, which is Intel HD Graphics 4000 512 MB. Clearly not enough. And I would guess that a secondary culprit is the processor.

I wish I wasn’t on such a budget, because although the size and weight of this 13 is great for when I travel, that 15″ retina model for almost three times the cost really edited my footage better.

-Trevor Ward
Red Eye Film Co.
http://www.redeyefilmco.com
Orlando, FL