Besides running all updates, i would make sure not have more than 10 clips max.
Close other running apps, and do you have any 3rd party video cards? AJA, Blackmagic Design?
Try downloading the plugin again. If you havent already. I know that i had few issues on a 12-Core, when transcoding both REDCODE and 5D footage. The AJA plugin was conflicting and causing system crashes.
I hope you were able to fix the problem.
*Production*Post-Production*
Apple Certified Instructor (Final Cut Pro 7)
“Semper Fi USMC”