April,
AE will only render what is visible in the composition window. HOWEVER, it does need to keep track of other objects off-screen (so it knows where they are and if they need to appear in your video). This will start to eat into your render time, especially if you get very large, complex 3D scenes with a lot of objects in them.
In my experience, I clip layers that aren’t visible and that seems to help with render times. Once a layer is out of view (or off screen), I trim the out point to the last visible frame. This tells AE that it doesn’t have to worry about that layer anymore. Of course, the problem comes when your objects appear and disappear from the screen. You can’t set multiple in/out points along one layer. You could duplicate the layer, but that may be causing more work than you need.
Hope that helps a little,
Jacob