Reducing Export Time and relationship to Preview Renders

[Vince Becquiot]

Hi Keith,

In your export settings, you can choose to use preview files. In CS5 it’s a check box, in previous version it was part of a contextual menu on the right side. While this will vastly speed up your render if your timeline is already rendered to preview, it will essentially compress twice and that will affect quality.

H.264 looks good, but it take time to render to it. More effects like Magic Bullet can drastically slow things down. More processors and some newer GPUs can accelerate that process.

Vince Becquiot

Kaptis Studios
San Francisco – Bay Area

[Keith Moreau]

So, if I was exporting to Prores 422 1080 30P, and I had my sequence settings to that, and wanted to output a file of the same type, would the ‘preview’ files be in Prores 422 and then if I exported the sequence and had the ‘use preview files’ checkbox checked would it just basically copy over the files and not do the double-encoding?

I’m still trying to figure out it sequence settings makes any difference at all either in working with PPro timeline, speed of creating previews, or in the export process. The information out there is scarce, it seems, on this subject.

Thanks much for any advice.

[Vince Becquiot]

Not sure in the case of Prores, but for most formats you would re compressing existing preview files.

Vince Becquiot

Kaptis Studios
San Francisco – Bay Area

[Tim Kolb]

You set up the codec you use for preview in the sequence settings. It sounds like you’re on a Mac, so i’m not sure what codecs are available to you on that platform. Of course, only owners of Final Cut Studio can encode (or export to) ProRes…as the encode end of the codec isn’t installed on your Mac unless you own FCS.

Premiere Pro has a different operating model than Final Cut and it’s probably worth the time to note it…

Final Cut transcodes everything to the codec the timeline is based on. If the ingest format isn’t QuickTime…then everything on that timeline is the same codec and any preview rendering necessary is done back to that same codec. At the same time, any “rogue” media that may not fit the native codec/framesize/frame rate, etc. that you place on that timeline requires rendering.

In Premiere Pro, the sequence presets are largely for convenience. PPro will decode anything that you use that is compatible with a codec installed on your machine. You can mix any formats you care to on any given timeline up to the capacity of your machine to decode and buffer them, but even a modest machine seems to be able to handle same frame size/same frame rate mixing pretty well…XDcamEX, XDcamHD, XDcamHD422, DSLR H264, AVCHD, AVC Intra, and ProRes all seem to work on the same 1920×1080 23976p timeline with no real headaches when they are all the right frame size/rate…and I’ve even mixed up the rate a bit here and there.

When you mix all these formats together, obviously a transcode to a uniform codec on the front end a la ProRes and FCP, would affect each media type differently, and it creates an interim conversion that will lose some image quality, small as it may be. Premiere Pro’s design intent is to work with preview renders that are fast and as responsive as possible (I frame MPEG2, DVCProHD, etc) during editing, and since the timeline itself is not “native” to any codec, the preview codec is a setting that is user-controlled in the sequence settings, and is really not tied to any particular codec based on the sequence settings themselves.

When you choose to use the preview files to feed your Media Encoder transcode to your ultimate distribution file, you are using those preview files as the source. If you are editing a DVCProHD sequence, but have I-frame MPEG selected as the preview, you’ll be compressing your H264 or MPEG2 (or whatever) distribution file using the MPEG preview file as the source, not the DVCProHD source material.

In order to use that original source material, the effects and transitions, etc have to be re-executed in the Media Encoder. This can take some time, and this is another reason why a very robust machine is best for running PPro.

In the cases where there are effects that aren’t quite CS5 “ready” or somehow have issues being deployed in the Media Encoder (actually a second PPro is running behind the Media Encoder to frame-serve to it) when it’s accessing the sequence, checking the “use previews’ box after you’ve rendered a preview can be the only way you’ll get the output to actually use the effect as the preview already has the effect “baked in” and Media Encoder (and PPro “Headless” which is the background app that is tending the sequence for Media Encoder) doesn’t have to re-execute it…

So…obviously when you use the preview files, you bypass needing to re-execute all the effects, etc on the sequence while encoding…however your “original” is now your preview render. Not necessarily a bad thing…it’s just best to understand how it works, why it works that way, and what advantages and compromises there are to each approach.

Once you move to formats with color precision above 8 bit and you start to use “maximum bit depth” and “maximum render quality” for the sequence and/or the Media Encoder, you add another layer of decisions to your workflow.

I think many users coming from FCP may find the decisions they now have to make a little daunting as FCP really doesn’t have as many variables in its workflow portfolio.

However, as time goes on and the flexibility and capability of CS5 is understood, the ability to work with native files on the harddrive as opposed to ProRes (XDcamEX HQ= 35 Mbits/sec vs ProRes 422 at 220 Mbits/sec) freeing up harddrive space, allowing harddrives to actually deliver more streams (at least up to the ability of the installed CPUs to decode them, and being able to work with higher-end workflows with deep color precision and features like over-and under-range colors that can be stored in DPX frame sequences for use in other stages of production…will become more and more appreciated as our industry continues to become more diverse, and the demands placed on us continue to increase in complexity.

TimK,
Director, Consultant
Kolb Productions,

[Keith Moreau]

Thanks Tim for taking the time for this great explanation. I guess if you have the time to wait for an encode to be done, than it is best to retrieve the original media instead of using the preview files.

However, if you don’t have the time to wait, or the need for ultimate quality for a particular situation, it might be good to have a good preview setting just in case you just don’t have time to re-render everything. I’m still not sure what the Sequence’s “Editing Mode” settings are really for, other than framerate and framesize, maybe they don’t really mean anything.

From what I understood from your explanation, the sequence setting codec ‘Preview File Format’ would be what I would render from if I needed to ‘Use Preview Files’ while exporting. Therefore if I chose a very high quality preview file format, I might be able to utilize this as the ‘source’ if necessary and not have to wait for these effects to be re-rendered. However if the codec is too CPU intensive it could slow down editing and previewing, i assume.

As an experiment I’m going to try to set my ‘Video Preview Files’ to Prores and see if this bogs down my system and see if i can get fast Prores exports.

Thanks again for your explanation. I don’t think this information is readily available anywhere else.

[John Frey]

That is a great explanation, Tim. Your input is greatly appreciated.

John D. Frey
25 Year owner/operator of two California-based production studios.

Digital West Video Productions of San Luis Obispo and Inland Images of Lake Elsinore

[Tony Sarafoski]

Keith, I’m curious to know what your outcome was by setting your preview to ProRes?

Being a long time user of FCP, I decided to give PPro a crack after a colleague showed me how he could cut native H.264 files from a Canon 5D MarkII. I was so gobsmacked that I decided to give PPro a go on my next edit.

Having already captured my rushes as Apple ProRes 422 via FCP, i down-converted the ProRes 422 files to Apple ProRes 422 (LT) because I was going to attempt using PPro on a MacBook Pro 17inch i7.

Only just completing the edit today, I began preparation to master the final edit to MPEG2.

I mentioned i was gobsmacked at how PPro could handle native codecs, well i was yet again gobsmacked when the estimated time to encode this 2 hour timeline is telling me that its going to take a little over 7hrs to encode.

Please tell me I’m not going crazy nor seeing things, but 7hrs…? The clips on the timeline have “NO” filters, no third party Plug-ins & are as raw as they come (ProRes 422 (LT).

I was hoping to speed up my process, not add hours to render times.

I also tried MPEGing another sequence that had a few MB filters skatered throughout, and it’s telling me that, that sequence is approximating just a little over 13hrs to enode to MPEG2.

I’m feeling very confused at this stage…!

[Keith Moreau]

This is a response to Tony:

I’ve had the same unreasonably long encoding times as you. It’s perplexing but without data to the contrary, I think this is the way it is with PPro and the Mac.

I covered a lot of this in another thread here, https://forums.creativecow.net/thread/3/909117#909117, but so far my investigations have led me to believe that on the Mac, even with a powerful Nvidia MPE approved card (Quadro 4000) the Adobe Media encoder, while it may be producing great renders, takes a really long time, especially when scaling the footage.

I have not found any time savings to using the preview files as the output encoder settings. The encoding times are the same whether I use ‘use preview files’ or not. I haven’t seen any benefit to using Prores as the preview format.

In addition, at least for me with a 4 Multicam sequence with keying, color correction, garbage mattes and multiple tracks overlaying, I’ve found when I downscaled from 1080P to 270P (iPod sized) the output had strange lines at the edges of white frames that were within the frame. Hard to describe, but these edges, which should have been invisible, showed up as light fuzzy gray lines in the down-rezzed output but not in the preview, nor in the full-rez output, which makes down-rezzing unusable to me. Outputting at full res didn’t produce these lines.

The very fastest workflow for me was this:

1) Set the output to match the sequence settings. This blanks out all the settings and should produce the equivalent of FCP’s Export ‘match sequence settings’ Quicktime reference file. However this is nowhere near as fast as FCP’s reference file output.

2) Uncheck ‘use max render quality’. For low rez deliveries where it’s non critical I don’t know if this setting matters and if it’s checked it doubles the encoding time

3) Output the file by queuing it into Adobe Media Encoder. In my case my sequence was set to Prores LT 1080P, so it out that type of file. The encoding time, for me is about twice that of the sequence length, ie, 1 minute of sequence = 2 minutes of encoding time.

4) If you need a faster down-rez and/or re-encode to something else, use Apple’s Compressor/QMaster combo or Quicktime Player 7 on the PPro QT output file. Using QT7 or Compressor, you can output in faster than realtime, so the whole thing is about 3x the time of the original file.

I’m going to see how using the native MPEG2-Iframe sequence settings as the original sequence works with this workflow.I don’t know if Quicktime tools can handle this codec, we’ll see. Probably Mpeg Streamclip could. I agree with you, I can’t afford to have a day of Media Encoding encoding on a short project. I really can’t accept artifacts in downrezzed stuff that media encoder is producing. I can’t use the output. If anybody has advice for us to use Adobe PPro from beginning to end on the Mac for faster encoding times as well as avoid this artifiacting, let me know.

Hope this helps, it’s great news, though for me the real time aspect of PPro using native files, is still a great benefit to me and I’m not going to throw it away, I’ll just have to deal with it and hopefully hit on an optimal encoding scheme.

[Tony Sarafoski]

This is a response to Keith:

I’m thinking there must be a glitch with the MAC version of PPro CS5?

As mentioned in my previous post, a colleague that introduced me to his PPro workflow, certainly isn’t experiencing render times like this. From what I recall he is running a PC with a 3ghz core 2 duo, 8gig Ram & a Nvidia graphics card that is mercury friendly.

Having read your post I decide to re-captured x2 HDV tapes (using PPro) in the I-Frame MPEG codec.

Here is a snapshot

I then imported all the rushes and dropped them on the sequence icon to create a sequence that conformed to the rushes.

After adding my IN/OUT render markers, I selected FILE/EXPORT and chose the following settings.

Keep in mind…! I captured the HDV tapes using PPro in the I-Frame MPEG codec (which is the preferred code PPro likes to use) and I didn’t add any transitions, filters or effects.

And this was the estimated render times on the latest MacBook Pro 17inch which has a 2.66 GHz Intel Core i7 with 8 GB 1067 MHz DDR3 RAM.

As much as I’d like to edit native files, these render time are ridiculous. It’s back to FCP for me….!

[Keith Moreau]

This to Tony:

Tony, I didn’t follow up with the Mpeg Iframe codec. I did exactly what you did, using the ‘native’ premiere pro codec, I tried to encode using the iFrame codec and it didn’t help at all with render times. I then tried exporting the iFrame file using the ‘use sequence settings’ and got a file output (which was very large, like a prores file.) However nothing I tried on the Mac would recognize the file, including Media Encoder. Apps might open it but it was black video.

However, my workaround was to set my sequence to Prores LT (but any prores would work), and the output a file using the sequence settings, with “use Max render quality” to off. Then I take this into Quicktime Player, Compressor, or mPegstreamclip or even Adobe Media Encoder to trancode and scale. All these are pretty quick, including Media Encoder. You can check this or my other threads for details.

I think there is a bug in the Premiere Pro to Media Encoder connection on the Mac. There is no way this should be this slow. I think there are a lot of Mac users using PPro and Media encoder, they can’t all be having unreasonable encoding times like this. Adobe? Can you test this out? Sounds like Tony has a very simple HDV workflow, it would be easy to test… Adobe, are you listening?