Well, first off you DON’T want to use H.264 as you’ll be double-compressing your video. Once to H.264 and then again to MPEG II once you put it on the DVD.
You want to go directly from Final Cut to MPEG II. The best quality involves picking EXPORT -> COMPRESSOR right from Final Cut. That will tie up Final Cut while dragging a QT movie into Compressor doesn’t, but you’ll get slightly better results.
You should probably be aiming for a VBR MPEG II compression with an average of 6.0 and a max of 7.2. Also, make sure you compress your audio into a Dolby 2 file in Compressor too. You’ll then have an audio file and a video file that you can import into DVD SP.
That should look pretty good…once you get a disc made you can look at the size of it. If it’s less than 4 GB, raise your average from 6.0 to something higher and try again. Don’t put the max much higher than 7.2 or 7.3, however.
Also remember to judge the quality of your DVD on a TV and stand-alone DVD player…not your computer screen.