Smooth out the peaks with volume automation or som kind of software (have not tried Levelator). Traditional compression can make the sound even more harsh. Use an eq and try to identify what frequencies are causing the “brittleness” and try lowering them a bit. You mostly have to compromise in situations like these.