You don’t give information about your clips, version of Premiere, or hardware/drivers on which to base a detailed answer, however…
Small size changes have been noted with some capture systems, mostly with pre-Pro versions of Premiere, when a clip is rendered. Unchanged sections of clips are not affected, so the change to/from a rendered section can cause a slight ‘jump’.
The ‘trick’ to work around this issue, which is generally attributed to the capture card and associated CODECs rather than Premiere, is to apply a ‘do nothing’ effect to the whole clip to mask the size change.
‘Do nothing’ effects include Color Balance (HSL) with no change from neutral.
The cost is that more of the timeline may need rendering, but the benefit is to ‘fix’ the problem you may be describing.