You hardware specs are really low for so much decompression overhead on your source footage. There are reasons why the i5 is not branded an i7 or i9.
Like the CPUs, there are different versions of the NVENC that you are looking at Vegas to take advantage of. The 680 was a mid tier gaming purchase in the 1st place. https://en.wikipedia.org/wiki/Nvidia_NVENC
I understand your argument if VP13 can edit those formats, why is VP15 struggling. With VP15 it sounds like vegas is attempting to use NVENC hardware for both encode(render) and decode(timeline.) This is a dramatic shift in from the way VP13 handled timeline footage. more than likely the 680 is over loaded with tasks, or the software is just not optimized for that generation of hardware and drivers. VP15 was most likely tested with the latest gear and not that heavily tested with legacy gear.
I would read some of the many posts on optimizing your pc for vegas. Also look up the VP15 minimum hardware for 4K and then improve on that by double if you can.