The right answer is indeed that it depends on your source footage and how you are using it. You should analyze your workflow before you can determine what the bottleneck might be.
To answer your specific question and give some info on hardware in general:
Within Premiere Pro, the GPU is mainly used to accelerate certain things. Primarily the use of ‘GPU-accelerated’ effects (those have the icon of a folder and play symbol next to them). When using those effects PPRO can leverage the GPU to maintain real-time playback. The GPU also accelerates export times greatly and can only help decode (and so play back) certain RAW formats like R3D.
The CPU is still the main driver of all things encoding (exporting) and decoding (playback). Certain codecs in particular are CPU heavy as they are heavily compressed: like e.g. H.264 or HEVC. That requires the CPU to work harder to decompress them. Others that are less compressed (ProRes, DNxHD) require less CPU power and can for instance be used on laptops more easily. H.264 in particular also benefits enormously from certain CPU technologies like Intel’s QuickSync, a tech only found in the mainstream line (and not for example in Xeon or Core X CPU’s). It is notorious how for instance a regular i7/i9 iMac will completely outperform a Xeon-based iMac Pro using H264 footage.
On top of that there is hard drive speed as an important factor. The less compressed formats (ProRes, DNxHD), especially when in higher resolution, require drives that are faster than ‘normal’. That means SSD’s or RAID systems. Compressed formats like H264 have smaller data rates and can work fine on slower spinning drives.
As you can see: depending on your source footage you need a very different situation to get optimal results.
Doing a lot of H.264?
Get a ‘top end’ mainstream 6-8 core CPU (i7/i9), never do XEON. Fast drives are welcome but not your primary concern.
Doing a lot of ProRes or other optimized?
You will do fine with professional CPU lines like XEON or CORE X, especially at 4K or above. For normal HD resolution you can go with cheaper mainstream CPU’s. Fast drives like SSD and RAID may be necessary depending on resolution and complexity of the codec (ProRes 444 vs Prores 422).
Looking at your specs: the XEON is certainly the culprit if you are trying to work natively with H.264 footage (GOPRO, DSLR’s, etc.). If you are now just struggling to playback real time effects and lumetri etc., getting a better GPU like the RTX series or 1080Ti will help you for sure. They will also help if you are doing RED RAW. Or maybe you are using regular spinning disk to play back high data rate codecs (ProRes 444?) or RAW?
Good luck in diagnosing your workflow, hope you find a good solution!