Taking a mask as the first approach: add three points at the top, one in the middle and two just off to the sides. Then set a key frame for the mask path, move ahead in your transition and drag down that centre point to the bottom, take it off screen completely. Move a few frames further on and move the other to points off left and right, so the mask opens up more.
The trouble with this approach is that you will cut the screen, your zip tracks(?) won’t move off, so you’d have to animate these separately. If you use a block shape layer, with repeaters for the keys, you should be able to create a flexible zip which can be animated to pull back.
Alternative approach: create a black and white gradient, like an inverted triangle, but tapering. So that it looks like a zip half opened. The use a gradient wipe of the image AND use it as with a gradient displace effect to distort the image at the same time.
http://www.YouTube.com/ShiveringCactus – Free FX for amateur films
https://shiveringcactus.wordpress.com/ – FX blog