First things first, what version of premiere are you working in?
Without seeing your project the best I can do is give you some suggestions for a solution which may or may not work.
If this is just one long continuous video, it looks like live capture game play, and not multiple clips you might want to just export the entire video at real time with the timecode baked in without nesting it. Then pull that export into your project and speed up the sections where you’d like.
Alternatively, have you tried not nesting the timecode to the video and speeding each one up individually with the same settings?
Nesting certain things like text layers, and it appears timecode, can cause some bugs and issues. This type of nesting is generally better done in After Effects since that is more of a graphics based system rather than Premier which is an NLE.