Premiere Pro equivalent to “Quicktime Reference File”

[Tim Kolb]

[Keith Moreau] “I did some tests using Prores LT as the preview file format, rendered the entire sequence so there was a green bar throughout the timeline, rather than yellow or red bars, and did encodes both using these preview files and without the preview files.

I found that there was no significant difference in the encode time either using preview files or not, even when I exported using the sequence’s format – in this case Prores LT”

Well…now you’ve changed the game somewhat by setting up your sequence this way. It’s not a bad thing, but I’m assuming the source material was ProResLT? When the preview files are also ProResLT, there would only be whatever you’d gain from pre-rendering effects, so I would guess the effects that you used in the sequence are either pretty minor, or they’re effects that are easily executed by the background PPro application that frame serves to the Media Encoder…

ProRes is a nice, light decode for most systems, which is nice, but I would bet that the fact that it, like most of QuickTime, is still 32 bit and that the 64 bit Adobe applications have some sort of an “adaptive” code fix somewhere in the background that enables CS5 to work with QT at all probably does create some additional processing over using the FCP>Compressor workflow where everything is 32 bit and it’s still completely compatible with the codec.

I might suggest, strictly for testing purposes, that you set the preview codec to DVCProHD P2. Yes, it’s a loss in color precision from ProRes, but you’ll be using a codec that is not dependent on QT and then I would be curious what type of performance you see…

As nice as ProRes is, keep in mind that while Mac ported to a 64 bit OS years ago…the NLE package and the media architecture really hasn’t changed much. I think that most software manufacturers embrace the Mac as a user interface of course, but making software that increases in capability as fast as the customer base would like while still supporting a file architecture that is nearly 20 years old at it’s core, is a real challenge.

Moving to 64 bit while trying to make 32 bit codecs work is no picnic either I’d bet.

TimK,
Director, Consultant
Kolb Productions,

[Keith Moreau]

Thanks for the reply Tim. In fact non of the source media was Prores LT, it was a variety, XDCAM EX 30P, AVCHD 1080 60P (Panasonic TM700) 24P AVCHD.

One thing that I just tried again when encoding another sequence to Prores LT (this was a really long 1 hour sequence) I unchecked Max quality render. This encoded to Prores in relatively a lot less time than my earlier render, at 1 hour 39 or a ration of 1:1.4 rather than about 1 hour to 6 hours of encoding. I think the max quality may be doing lot more to the codec. In addition I didn’t have the ‘Max Quality” checked on in my preview files setting. Maybe they need to match. I’m going to check the ‘max quality’ and render to ‘max quality’ using preview files in the encode and see what happens.

I’ll also see if the DVCProHD P2 format helps me out more.

Thanks I’ll report back with additional findings.

[Keith Moreau]

OK, more tests. DVCPro HD P2 as the preview file format / export format didn’t help the encoding time at all, in fact it was 30% slower than Prores. I also made sure my ‘use max render quality’ matched both the sequence preview file setting and the output encoder setting. It didn’t make a difference whether I used preview files or not. The encoding time did vary greatly based on the use Max Render Quality Setting, about twice as long with that setting on.

So, at least in my preliminary findings, there really isn’t a way to speed up encoding by using preview files, therefore I don’t think preview file setting is important at all. I think I’ll go back to using the MPEG I-Frame format as I think that’s probably the most efficient for PPRo to deal with (though who knows.)

I think this is a great room for improvement here to help with encoding times. I think PPro needs the equivalent or even on the Mac platform, use the concept of Quicktime reference file. I was reminded how useful and space efficient they could be when today I helped a fellow filmmaker encode a 720P video for Youtube upload. We exported the Quicktime reference file in about 15 seconds, and then sent that to Compressor and used a “Cluster” to max out the CPUs and it took about 10 minutes to encode a 6 minute video. My feeling is that PPro / Media Encoder would take about 4 times longer than that. (though probably output better quality.)

I’m still going to keep searching for a workflow where I can somehow make this encoding process more efficient, short of upgrading my CPU. I’m still not convinced that the GPU is being used in encoding, but I’ll probably do some more tests with the GPU off, if that is possible (setting MPE to not use the GPU).

If anybody has any more advice for me, please let me know and thanks to Tim and others for your suggestions.

[Tim Kolb]

Max Quality is most useful when outputting a down-resolution encode.
(1080 to 720 or HD to SD, etc). It certainly does slow down the process, however in my experience it creates a very high quality result.

Depending on how many clips are in your timeline, I would go into the Media Encoder preferences and tell it to not use Metadata at all…that might help.

When you tried the P2 files, how did you do that? You didn’t load FCP-ingested P2 files did you? That would mean you’re still using QT on the front end…which i am quite certain is part of the choke.

FCP, ProRes and Compressor are compatible with 32 bit QT and PPro CS5 is making an adaptation to even be able to use a codec/media architecture that is 32 bit in a 64 bit environment.

I don’t think that using a reference file will help anything as you would still be accessing the same QT files at some point…and I’d bet that QuickTime has some culpability in this as I have an older Windows machine that seems to perform better than what you’re describing, even with QT files, though Adobe had to write some more proprietary ways to handle 32 bit QT on the Windows side…I’m not positive they even have the latitude to do that on the Mac side.

I think Mac users really do need to note that this isn’t Adobe’s problem alone by any stretch. I Apple users might ask why the computer company that exclusively went 64 bit in its own proprietary computer platform just a couple months short of a decade ago with Cheetah has only really just started on moving its media architecture forward (QTX), and even that still relies on a host of legacy 32 bit components at this point. (Keep in mind that I am a former exclusive Mac user as that was the “media savvy” computer for a long time…)

Getting equal performance for a 64 bit media application between Mac and Windows requires Apple to advance their platform too…Adobe has had 3 full version releases and 9 significant update patches since the last time Final Cut had a major code revision. (They’ve added lots of very nice features…no question, but universal binary and dropping AE plugin support was the last major restructure.)

Hopefully “Lion” will make some advances for Apple in some of these areas…if a really substantial QT overhaul is included.

Of course, if they do advance QT, it’s hard to say whether they’ll give Adobe any time to prepare as the corporate arm-wrestling match continues…

(sigh)

TimK,
Director, Consultant
Kolb Productions,

[Keith Moreau]

Hi Tim

In the case of my tests, the media files were native files, XDCAM EX with an .MP4 extension, and AVCHD files with a MTS extension. No quicktime wrappers at all.

To change the preview files format to P2, I went to the sequence settings containing the above clips and change the preview files setting from Prores to P2, it warned me that I was about to delete all the preview files if I changed it, I accepted that.

The sequence then had a ‘yellow’ bar instead of green. I then rendered the entire work area, eventually the yellow bar turned to green. I then attempted to export using the P2 format, which was slow.

I hope my posts are not disparaging of Premiere Pro. As I use it more and more I’m finding I can do almost anything I did with FCP, and more. The reason I’m ‘switching’ is that I can get to work more quickly than I could with FCP because of PPro’s ability to work with all kinds of native formats. It’s really the leader here, even better than Avid in many cases.

However, at least for me, the one area that I haven’t found an efficient workflow is in the encoding process, specifically in how FCP / Quicktime could use Quicktime reference files, seemingly very efficiently compared to PPro. When I’m at the end of my projects, I’m just going to have to allow for a very lengthy encoding process, it’s not a dealbreaker, it’s just something I’ll have to deal with.

I’m also very concerned that the Quicktime architecture needs a major facelift, and because QT has so many tentacles into the Mac OS and Final Cut Studio, it’s a lot of work for Apple at a time where their ‘Pro App’ focus is probably not what it once was. I have a feeling it will be 2012 before this issue is resolved, if at all. In the meantime I need things to work. The last major feature upgrade was in 2006, and I’m continually working with editors that are still using that 5 year old software. At one time it was amazingly ahead of it’s time, now it’s outdated in many ways, especially in the need for Quicktime media.

I’ll try the ‘metadata off’ thing and see if it helps, I may also just need to upgrade my Mac, it is an early 2008 8 Core Mac, and though very serviceable, is probably 1/2 the speed of the fastest current Macs. I’m not ready to switch to a PC for PPro, though the though is attractive, too much other aspects of my work is Mac-centric and it would be a huge change (though I do have to work with PCs all the time.)

Thanks again Tim, for your time and expertise, I really appreciate it and I will continue to keep this forum posted on my findings.

[Tim Kolb]

[Keith Moreau] “I hope my posts are not disparaging of Premiere Pro.”

I think it was the “I think PPro needs the equivalent or even on the Mac platform” statement that grabbed me actually.

There seem to be many that imply that whatever differences there may be in speed or processing power between the Windows and Mac platform is solely Adobe’s issue and that they better “get with it”…

I’m not advocating for Windows (there are aspects of Windows I absolutely detest…and aspects of Apple’s OS that I think are brilliant) or saying that nobody should criticize Premiere Pro…(ask Adobe…I get in my share of “suggestions”) and I’m certainly not saying everyone should dump their Macs and get a Windows machine.

All I’m trying to do is to illuminate some of the roadblocks Adobe has on the Mac side of things at this moment. I think for the most part, they work around a lot of it pretty well, but I think some development on Apple’s side will help immensely.

While your machine may not be new, it shouldn’t be terrible. I’m not an expert into what was installed in what Mac when, so I don’t know if you have 8 hyper-threading (16 logical) cores, or just 8 physical cores. I also don’t know what the relative threading limitations may be for applications as you go back in time in hardware and OS. Can Media Encoder use all 8 cores in that processor? Under your particular OS version? If it isn’t using all 8 cores, then I can see where that won’t help much in the speed dept.

The “clustering” you’re referring to sounds like multiple iterations of the Compressor application opening to work on a job… Depending on how well Compressor utilizes multiple threads, maybe multiple application processes is the only way they harness all the CPU cores?…it’s what we used to do with Canopus ProCoder back in the day before it could multithread…

If you have a CUDA-compatible display card, The CUDA cores will help with certain things like scaling, etc., but video codecs (decode or encode) aren’t written to run on GPU cores…they’re all CPU. If you’re a little limited in the CPU dept, that won’t help either.

All that said…what format are you compressing to? What is the edit timeline like…number of clips…number of effects…?

Also…what is the type of media you are using and the sequence settings you are using it on? Is there some inherent scaling on every clip? Pixel aspect change? You should see some difference in encode time if you have lots of effects by rendering, and sourcing from, the preview files…whatever they may be…because those effects just simply don’t have to be recalculated.

I’d like a clear idea of your total configuration, media, sequence settings, encoder output specifications, etc…there has to be some piece of info that will shed some light on the situation that we’re missing.

TimK,
Director, Consultant
Kolb Productions,

[Keith Moreau]

[Tim Kolb] “There seem to be many that imply that whatever differences there may be in speed or processing power between the Windows and Mac platform is solely Adobe’s issue and that they better “get with it”… “

I certainly didn’t imply this but if it seemed so than I apologize. I think at this point that Adobe is doing a lot better job in the NLE world than Apple, that’s why I’m switching from FCP to PPro.

As far as if my CPUs are being utilized to the fullest by PPro and Media Encoder, they are all at 100% when the encoder is running. I like the way it uses the cores and is very ‘gentle’ about backing off if another app needs some time. Much better than Compressor / QMaster in my opinion which pretty much disables the computer until it’s done.. My 8 core was prior to hyper threading and this is a probably one reason for the slowness.

I have a Quadro 4000, which is supposedly a pretty good card for PPro / Mac MPE performance. Previously I had a Quadro 4800, then returned that and had a GTX 285. I’m still not sure if the 4000 is faster than the 285, but it seems a bit more ‘stable.’ I’ll be testing both soon against each other.

Most of my tests were done on a 13 minute timeline, with really just a several long clips, but extensively edited up into hundreds of edits, with 3 types of media, XDCAM EX, AVCHD 60P, and a H. 264 Quicktime movie (an iShowU screen capture.) There was Ultra Key on the EX and AVCHD, as well as a fast and 3 way color corrector on all the clips, and an occasional gaussian blur. All the effects were GPU enabled and I could play them real time, if I didn’t render, the timeline was all yellow. No red. On the AVCHD there was some severe scaling and rotation on one video track, I tilted the camera at 90 degrees against a green screen and scaled it.

I encoded primarily to Prores LT, but also encoded to the “Apple TV” format, which is simply 1280×720 30P H.264 at 5000kbs.

At this point if I need my fastest encoding workflow I will output a sequence with the same codec and settings which seems to be about 1:2 (1 minute of source takes 2 minutes to encode), then take that into Quicktime player or compressor and turn it into something else, which seems to happen a lot faster than it does in Media encoder with the Ppro sequence as the source. If I have time I’ll just do it all in media encoder, as that’s simpler and probably much higher quality.

Thanks much for this continued conversation and investigation.

[Tim Kolb]

Well…when it comes to general application H264, Adobe Media Encoder is getting better and better, but QT Pro still makes H264 output files that seem more widely compatible for whatever reason…

That said, I’ve been using AME to output Apple TV H264 and iPhone targeted H264 files for the last year or so and that seems to work out…

Now…

Your Quadro card choice sounds excellent…You should have very good timeline preview performance.

The problem here is that you should see a difference between encoding from the previews and encoding from the source files and re-executing the effects based on what you’re telling me about the content and effects of your timeline…in fact it should be relatively significant.

No matter what processor you are running, running an encode from the rendered previews should eliminate a lot of processor work for the encode…particularly with the long GOP MPEG4 and MPEG2 source media you’re using, and the effects just make it that much more odd.

Something is wrong with this picture.

Anything else significant going on with this project? There is simply something we’re missing here…

TimK,
Director, Consultant
Kolb Productions,

[Lilia Zar]

But i have very big problem-

I’m applying for art school. In the application form we got this instruction:
“Your film must be in QuickTime with maximum solution 720 x 576 pixels”

After I export it, my file wight 1 GB!! How can i reduce the size without change the QT format??

I’m using Pr.CS5 and my settings are-

format-QT
Codec-H.264
5 min
DV pal Wide screen 720*576
fram rate -25
field tips- lower first
Audio- 480 Hz, stereo, 16bit.

I tried to play with the rest of the settings (like Quality, key frame, bitrate settings and audio) in order achieve smaller file, but it’s insist to stay as 1.1 Gb exactly!!
Is it possible to stay in the setting with – Q.T format and to have file smaller then 300Mb?
Do i missing something in my settings?

The dead line is tonight
PLZ Help….