Definitely: cartoons have large blocks of color that are generally easier to summarize and encode to lower bit rates.
I don’t know of a tool that would do what you want, but it’s a great idea. I’m imagining a Photoshop-esque Before/After window with a sample of the compression. I do know that there isn’t an objective way to generate quality scores. Even PQA techniques that involve pixel comparisons that do give you an objective score often generate numbers that are unrelated to subjective quality analysis. For example, even if individual frames look good, it may be that the video looks like crap in full motion.
One pointer, though: cartoons with flatter colors and sharper boundaries often look better with “sharpness” turned up (at the cost of smoother motion). Not all codecs have settings for this, but if you have that option, that’s probably what you’ll want.
And one thing to watch out for: While solid blocks of color are easier to encode, they also reveal noise and, especially annoying, encoding keyframes. So if you start seeing a periodic “pulsing” in solid areas (the area will look perfect, then gradually deteriorate for a dozen frames, then snap back to perfect, and so on…), that’s the keyframes. You can add keyframes (Keyframe every 6 instead of 12, for example), but this is tricky and unfun to deal with.
Sorry I don’t have a shorter, better “Do this” answer, but I’d love to hear/see how your experiments turn out.
______________
A story should have a beginning, a middle, and an end…
but not necessarily in that order.
— Jean-Luc Godard
https://videopia.org