It depends on how good your balance is, how much time you're prepared to put and the usual conditions on the sea where you are.
The Elite as you know is a lot more tippy than an XT, but some paddlers cope with it; expect to take a few months to get comfortable on the Elite.
On the other hand the Swordfish is one of the most popular boats on the planet, so you should be able to sell it for almost as much as you paid for it, if you look after it.
My recommendation would be to go with the Swordfish. Look after it and you'll be able to sell it again when and if you want to upgrade to the Elite.
Rob
Currently Epic V10 Elite, Epic V10 Double.
Previously: Swordfish S, Evo II, Carbonology Zest, Fenn Swordfish, Epic V10, Fenn Elite, Red7 Surf70 Pro, Epic V10 Sport, Genius Blu, Kayak Centre Zeplin, Fenn Mako6, Custom Kayaks ICON, Brian's Kayaks Molokai, Brian's Kayaks Wedge and several others...