is the logo animated? if not, i would use photoshop… it sounds like there are some jpeg like artifacts around the edges, and if the colors you are trying to keep are close to white, you will have problems. photoshop has better selective blurring tools to help smooth those out.
if you don’t have photosop, the logo is animated or you just like to work in ae. you might try looking at the individual channels (r, g, or b) with the show channel selector in the preview window (colored circles button). if one looks better than the others, better contrast or less noise, then you could try and use that channel to create a matte.
duplicate the logo layer and use the channel combiner effect on the duplicate to make that color channel luminance only. add a levels adjustment to make dark areas black and the light areas white (white will be opaque, black transparent in the next step). set the original logo layer to use the duplicate as a track matte. if you get a white fringe, precomp those two layers and then add remove color mattting, with the color set to white.
Kevin Camp
Designer – KCPQ, KMYQ & KRCW