[Al Levine]

Nesting has always been tricky and buggy in almost every NLE. FCP 7, Avid and Premiere… Have you tried exporting the clip and brining it back in, instead of nesting?
Also, I believe Premiere 2017 requires at least Mac OS 10.11 — a few bugs can appear just by not updating your OS.
macOS 10.12 has been rock solid with my copy of 2017.
10.13 is recommended if you move up to 2018.

[Al Levine]

You should try the latest version of Premiere (through Creative Cloud). CS6 is pretty old at this point (built for OS 10.8 if I’m not mistaken), and you’re running 10.12. And Jon is probably right… You may be working with media that CS6 isn’t great at decoding, but the newest version probably is.

Premiere has improved IMMENSELY since CS6. CC 2017 is incredibly fast. And you can get it free for 30 days. Worth trying!

But if you’re adamant about sticking with CS6, let us know what type of media you’re dealing with. You might need to convert to a proxy for playback to work correctly.

[Al Levine]

I’m on the same iMac as you with latest version of Sierra and Premiere 2017 11.0.2 (one version higher than you), and have had zero problems. In fact, Metal support seems to work FAR better on Sierra than El Capitan with the latest release of both Premiere and MacOS. Definitely a safe upgrade (if you’re not in the middle of a project).

[Al Levine]

Seems to be a bottleneck somewhere. It’s hard to tell where exactly. Either in the OS (messy system, maybe you need a clean install), the hard drive or the memory.

Are you running a client monitor off an AJA card? What is your audio output set to? Options other than Built In Output often times can screw things up if not configured correctly.

Few observations:
• FW800 is pretty slow to be running media off of, but not impossible by any means. Maybe set it up to read off the FW drives but write (cache/render files) to your local drives.
• 16GB of ram is very low for a professional environment. FCP7 only used 4GB of ram, but Premiere will eat it all. Throw more at it. It’s cheap.
• We had an editor on our show running 10.9 having very similar issues. He upgraded to 10.11 and his system is screaming again. But install more ram first.
• Your video card is pretty old and premiere loves a good modern video card.

[Al Levine]

Try clearing your Media Cache Files / Media Cache Database and any render files.

You can also try exporting with Media Encoder instead of Premiere. Just press Queue in Export Settings. That always clears it up for me.

[Al Levine]

There will be a SIGNIFICANT difference if you upgrade your RAM to 32 or 64GB, and upgrade your video card (GPU). Maybe even an SSD boot drive or faster media drives connected with Fibre or eSATA.

Your bottleneck with Premiere is not your CPU, it’s the RAM and GPU.

[Al Levine]

Looks like you’re shooting with a low resolution / compressed format. More details would be great — what camera you’re using, what resolution, format, etc…

Little trick…
Back in the early 2000s when I would shoot green screen using a DV/HDV camera, I had a little trick in After Effects. DV format didn’t have too much color information, so the goal was to add color information that didn’t already exist to the image and then key.

In After Effects:
Create a new composition using existing footage
Duplicate the video track. Turn the first video track to a Color mode, and turn opacity down to around 10%. This adds more color information to the video track.
Then Precomp that composition and throw a Keylight filter on the new Precomp.
Your key should turn out a bit better…

[Al Levine]

You can always try doing the keying in After Effects – you have more keying options in there.

[Al Levine]

Try Spill Suppression in the ultra key effect.

[Al Levine]

Completely depends on what you’re doing.

I have both setups. At my office I’ve got a loaded MacPro because I’m dealing with thousands of files, a SAN, and extremely quick turn arounds. After Effects renders while simultaneously exporting a cut and referencing many many versions of a project.

At home I have a MacBook Pro and 27 inch monitor, and I mainly use it for small freelance projects. Small promos, personal projects, etc…

So it really depends how deep you plan to go. If you’re dealing with 4k, effects, massive SAN drives, quick turn arounds, etc… go with a MacPro. If you’re doing simple edits, some effects, but the portability is going to help you out more in the long run, go with the MacPro.

Middle ground: iMac. It’s sort of portable (office to office), but extremely powerful.