Without seeing the result, we can’t tell if it is what we might expect, or can be improved, but your end product should be as good or better than it was prior to the camera upgrade. Starting with MPEG4 should not be a quality issue. However, getting from their to high quality standard def MPEG2 might not be as easy as it should be. Frankly, most software encoders don’t do MPEG2 well.
Dropping to NTSC is a major loss. Converting to MPEG2 is not, as long you compensate with bitrate. MPEG2 requires roughly 2x the bitrate to achieve the same quality. For standard definition (480), a 9Mbps MPEG2 is maximum DVD quality, and equivalent to a 4.5Mbps MPEG4 (h.264). This is a subjective rule of thumb that really only applies to lower bitrates. At higher bitrates above about 10Mbps, the difference is negligible. The real difference is that MPEG4 can maintain quality at much lower bitrates than MPEG2. An MPEG2@20Mbps is (arguably) broadcast quality and should be visually lossless compared to a standard definition master. Modern systems should
have no problem playing a 20Mbps file.
The other major issue is interlacing. You do NOT want it anywhere. If you are adding interlacing at any stage, you may wreck the picture. In theory, this should not be true going from progressive to interlaced. In practice though, it can be bad. Many compressors simply don’t handle MPEG2 progressive well even if you specify it. Many
compressors will presume that MPEG2 is interlaced because MPEG2 was specifically designed to handle interlacing.
In my experience, few software compressors do a good job of MPEG2. The best MPEG2 encoding is hardware-based and expensive. I do not use Apple Compressor, Adobe Encoder, or Sorenson Squeeze for high-quality progressive MPEG2 because I think they are all wanting. I use a lesser-known and cheaper Windows-only compressor called TMPEG Videoworks by Pegasys. In my opinion, the legacy TMPEGenc encoder (which can actually still be found for free) is better and much cheaper than any of the name brands I’ve worked with.
Yes, you should upgrade to 1080p, at least, but in the meantime your outputs should be at least as good as they were.