So it seems to do the trick. Key is to not use the mouse too much to do scrubbing. Every time I scrub fast through the footage, FCP seems to crash. It’s a shame JKL doesn’t let you play 2x and 4x speed. Here are some useful keyboard shortcuts I found:
Ctrl + Cmd + (Left/Right) – Previous/Next Footage
F2 – Add Clip to Queue