It does sound a lot like ulnar nerve. You should be getting numbness of the pinky and of the outside (pinky side) of the ring finger but not of the inside. The nerve can get trapped at a number of spots including the neck, the elbow and the wrist. The most likely ways of doing it while paddling is overusing the muscles that flex the wrist and move the hand toward the pinky side; or exerting too much pushing force, putting pressure on your wrists. One way of addressing both of these is to concentrate on not cocking your wrists outward and allow your outside fingers to relax. You should be able to maintain a full power stroke without tension in these two fingers. You might have noticed some good paddlers even have their pinky fingers out, like holding a teacup daintily.
It is possible to overdo this: I managed to give myself a forearm compartment syndrome when working on my stroke power, resulting in tingling of the middle, index and thumb as well as weakness of the hand. This gradually got better when I backed off and went at the building up process a bit more gradually. I think my problem wasn't really wrist position, but too much too soon, so keep that in mind.
Good luck. If getting nowhere, go see a physio, preferably one who paddles.
Past skis: Spirit PRS, EpicV10Sport Performance, Epic V10 Elite, Stellar SES Advantage. Current skis: Fenn Elite Spark, Fenn Swordfish vacuum. Custom Horizon, Epic V7