Let me do everyone a big favor. I just went through hell and back trying to solve this problem on my system:
Athlon X3 440 @3.4ghz
8gb DDR3
ATI 4850
Windows 7 64
And I finally figured it out.
I recently put together a short film in Vegas Studio Platinum 10 and started getting these low memory errors once the project was about 2/3rds complete. I was able to get out renders at that point only if I reboot the program and immediately rendered. As the project was closer to being complete it would not render at all- giving an almost immediate memory error.
I tried all the fixes- setting the video preview ram to 0, rendering threads to 1, etc. Nothing worked. The prospect of losing my movie after the time I put into it was unacceptable. I ended up upgrading my Ram from 4gb to 8gb- no change. I removed the overclocking from my CPU, no change. I formatted my system drive- re-installed windows 7 32, then back to 64, and a clean install of Vegas each time- no change.
Disgusted with the situation I found no other option than moving to another editing suite- which led to the discovery. In order to move to Premiere I decided I would try to render out each track in Vegas separately so I could drop the rendered files into premiere for continued editing. I muted all the tracks leaving only the main video track to render, and Vegas got through it, rendering the video-only track with audio disabled to a lossless AVI. I then rendered each other video track and audio track like that separately. (Audio tracks were rendered as WAV files)
After this process was done I realized I could drop these rendered files into Premier or… back into Vegas! I did so, and voila- I could now edit, modify, and render my entire movie in Vegas very quickly, without a hitch. Sure, I lost some functionality- I couln’t modify my envelopes, my color treatment was now permanent, etc, but I had a workable project again.
The bottom line is this- Vegas can’t handle a complex movie project, full of envelopes, transitions, color treatments, audio plugins, etc. Once the file reaches a certain level of complexity the software chokes and pukes out the memory error (weak design if you ask me). By rendering out individual tracks you decrease the rendering load tremendously, and by re-importing the rendered files you drop the complexity of the project file way down, allowing further editing and rendering without a problem.
Give it a shot. Hopefully my suffering over the last week can save you all a lot of pain.