Are you attempting to use the GUI as your reference display? No external monitor connected via some form of Decklink or other BlackMagic interface?
In this case, there are still some possibilities, but the best you will ever achieve is a simulation.
Your results will only be as good as the software that you are using to characterize your display, and that display’s ability to generate intensities that correspond to 709 values.
https://calman.spectracal.com/user-guides.html
…can work with BMD Resolve to generate essentially a display LUT that will pre-distort the colorspace so that it will be 709-like. It is not enough to simply match the RGB corners of the triangle… the whole thing has to match, as you are finding out.
Over the years, there have been many threads inquiring about “wide-gamut” displays — the hope is that “bigger” is “better”, but what a grade environment is really trying to achieve is a tight match, and some interpretations of “wide gamut” are incompatible with what should be an exact gamut overlay. Fine for photography, not so good for motion picture and broadcast production.
jPo
“I always pass on free advice — its never of any use to me” Oscar Wilde.