Ah, so nothing too fancy, just a lot of pain in the a$$ coordination of the shapes between planes. Sometimes brute force is the way to go.
Another way you might be able to do it is by creating a cylinder at the handle’s general angle with a diameter just inside the narrowest part and using the emboss mode of the wrap tool to create your shapes. Then use the offset surfaces to trim as you’ve done. The advantage of that is you’d have all the shapes in one sketch (or maybe two if you couldn’t get the overlap right on the backside), but you might get some unwanted distortion as the shape bends around the cylinder. Then again, you get distortion anyway so maybe it wouldn’t matter.