I know this is about Illustrator, but it might still apply.
I found the answer to the problem I was having in Photoshop: One of the layers had a bounding box that was limiting another layer with a big blue dot that faded out as it approached the outer edge of the dot. The halftone fade was “boxed” off. I was using a big blue dot that faded out as it approached the outer edge of the dot. I wanted it as a background layer for an object within it from another layer on top. Hard to explain, but I’m trying…
The answer: Start with the object layer. Then add a transparent layer, and paste the blue dot on it.
Wa la! No more “boxing” off of the fading blue dot. 🙂