Hey Eric, thanks for your reply!
I have since finished it and flew out to present the installation. It came out really nice, and I ended up solving it in different ways.
The animation was basically of asteroids flying all around in different ways – rotating around the room, flying up and down, and flying towards the spectator (who stands inside the room surrounded by the projected walls).
Having the asteroids fly around in rotation and up+down I did by placing a Parallel camera thats 4 times the width of 1 wall, so it captures all four walls at once. Then I faked perspective by having different layers of asteroids fly side ways at different speeds (the furthest, the slowest) and at different scale (the furthest the smallest). That way I knew that after rendering, I can chop each wall and it would be a continuous flow between the walls, and I would have insane perspective distortions at the sides of the video.
The asteroids flying at you part I really couldn’t solve and I realized that, unless I’m missing something, there’s some major physical limitations in projecting that with each wall continuing the one next to it. So I just exported 4 different walls. The effect was still intense enough to not bum out the asteroids disappear at the corners on the walls.
Anyway, this is still in exploration as I intended to make it better and better.