Hi Luke,
For speed you need stability in first to apply power & good hull performance after.
The fastest intermediate boat it's the one you can drive with "confort" in any situation. If you have stability, you can push hard on the paddle and have the speed.
Dean Gardiner and Oscar Chalupsky do great performance in intermediate or beginner boat because they are great paddler.
Try the "new" generation of intermediate boat with long & thin hull and you find your way.
For the moment after a swordfish I find the fastest for me, the Focus, in downwind condition I can follow the elite boat, because I'm in "confort" in all condition of sea.
Regards, Sylvain