Yes generally SD downconverts take a quality hit, but I’ve gotten good results from Compressor. Keep in mind you are doing several things in this conversion: resizing, changing pixel aspect ratio, and recompressing with a new codec. Understanding what all of these things are doing, you can create an optimal setting to deal with these. I would say there are two ways to go about it, and I don’t know how much better one is over the other. One is to create an all-in-one encoding preset with adjustments for these. The other is to create a chain, where you have separate presets doing the individual tasks. This would be to take advantage of what optical flow can do for you, but also add a lot of processing time. Often I don’t have that luxury, so I go with creating one preset.
To deal with the resizing and pixel aspect ratio changes, you should have frame controls turned on and resizing set to Best. You are going from a thin-raster 1280×1080 image with HDV pixel aspect ratio to a 720×480 anamorphic image with SD 16:9 pixel aspect ratio. It’s rough no matter how you slice it, but “Best” resizing should understand and do the best job. Leave fields as same as source, deinterlacing set to fast. This will ensure that you keep the original fields intact and not deinterlace. For adaptive details, you can play with the detail level control, but I’ve never seen much impact with it. Definitely leave anti-aliasing off.
To deal with the codec, this will be MPEG-2, but you have to ask yourself how long your material is. To be honest, CBR encoding works well with shorter length material. I do a CBR encode at 6.5 Mbps for most files up to 30 minutes. That’ll fit fine and does a good job with the above frame controls stuff set. If not, your VBR encoding is really the only option, but use these same frame controls settings.
The other thing you have to ask yourself is how you’re viewing the resulting DVD. A CRT? An LCD? A computer monitor? Each of these will represent your footage differently. I would say the CRT would look the best, as its native display resolution equals the disc’s. An HD capable LCD would be slightly less nice, because it is blending the fields coming in to display progressively (as all LCD panels are progressive displays). Less sharp in motion detail, and less sharp because the resolution is much higher and therefore SD material is enlarged to fit. If you have the opportunity to view it as 1:1 pixels, then the tiny sized image on your LCD panel will look much better. And then the computer screen – well that’s variable in other ways, but at least can often show you 1:1 playback in a windowed player.