I’d suggest making a black color matte and using it to cover up the pixels on the edge. (Or just use the crop filter, I guess.)
This would have the same effect as what you’re doing now but without having to enlarge (and thus degrade) the rest of your image.
Of course, you’ll have a few pixels of black on the edge of the screen, but if this plays on a normal television that will probably never be seen anyway as it will fall in the overscan area. Is that what the video is for? If it will be on the web or shown from a projector then this might not be the way to go, but for normal TVs you should be find to crop it slightly.