You should always leave it on. “Compensate for Scene-referred Profiles” allows the After Effects color management system to properly handle media with a scene-referred profile (which includes most real-world camera formats). Without this option enabled, Ae will handle them as if they are display-referred, leading to a slight gamma error. I assume the option is only there so that people could render legacy projects with identical results after Adobe added support for scene-referred profiles.
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn]