you could try making the parts editable, Functions/Connect then bridging the parts together by hand.
You can also try building it completely differently (there is rarely only one way) by taking a cube and extruding and beveling to get the shape inside a Hypernurb. Also remember with any symmetrical object you only need to build half of it (makes the details in the handle (from the first picture easier to do).
Here’s a bad example done quickly and not right but with better attention to detail could be another way to do it, with the advantage that smooth transitions from one surface to another are usually easiest when the mesh is one piece. Just started with a Cube, did three extrudes, some Knife work and an interior extrude.