Vector your video

[Ryan Hill]

If you have illustrator, you could check out this tutorial:
https://www.creativecow.net/articles/rabinowitz_aharon/cartoon_look/index.html

Also, since you mentioned Photoshop, if all else fails, it is possible to output an image series from AE, run it through a batch process in Photoshop and then import it back into AE.

This tutorial uses batch processing to resize a series of photos, but it could also be used to apply any photoshop filter:
https://www.creativecow.net/articles/kramer_andrew/slideshows/index.html

Of course, it’s a lot nicer if there’s a way to do the filter in AE itself, but I can’t think of an AE filter that does the equivalent of that.