You might want to remove a graphics card and see if things improve, as Shane said. Recent versions should be able deal with dual GPUs, but you really should be using approved hardware and drivers to pull that off. You won’t get any benefit in Avid from dual GPUs, that’s for sure.
A couple of other things to keep in mind….
Avid really hates having windows and pallettes over the source and record monitors. You can get weird refresh artifacts and poor playback, even if things overlap a little.
In addition, when Avid is transcoding in the foreground, it pegs all the processors on your machine and will use all available ram. Trying to do anything else in your machine will be difficult. The latest version has background transcoding, which is slower, but less demanding on your machine.
Lastly, it looks like you are not using an Avid approved configuration, which makes troubleshooting a bit challenging. Doesn’t mean it won’t work…many people do. You might want to look at the approved Nvidia drivers for Avid and make sure that’s what you have installed. If you are using other programs they might require newer drivers than Avid supports, so you might be looking at making multiple boot drives to get around this.