Hey Newkiter,
I am an ex-windsurfer from way back, did my penance of about 17 years on the sailboards and was addicted to it like crazy to the point of suddenly faking illness to go home from work everytime I saw the trees dancing (boss cottoned on after a while

)
Point is, I spent too many days sitting on the bonnet of the car waiting for those elusive two or three extra knots that wouldn't come, and then i'd go home after another wasted weekend sun burned and pissed off.
I bought a kite in December 1999 "just for those light wind days". That started a passion with kites that runs to this day. I ended up with a quiver of them within a year. My sailboarding gear was relogated to the back of the shed and accumulated dust. After 2 years I decided to sell all my wind surfing gear (2 new boards, 4 new sails, new booms and assorted accessories) and got next to nothing for it. A catastrophic loss.
I have now kited for a little over 6 years and have jumped on a sailboard twice in that time and both times I am reminded why I love kiting. Compact, lightweight, 3 Dimensional fun, easy to learn, ride in as little as 9 knots and up to about 35 knots with ease, fast, oh and nice airtime, and a huge array of tricks and styles of riding to choose from.
Chuck a surfboard into the frey and head to places like Gnaraloo and kiting, IMHO, is more fun and more challenging than sailboarding in the waves because the gear hasn't evolved enough to make it fluid which is a great because it represents a real challenge to try and get more fluid which requires great technique and really sharp skills not to mention the right gear.
I ain't saying poling isn't fun, it is .... in the right conditions (ie Gnaraloo). But flat water blasting doesn't do it for me and I'd never go back. Many broad minded sailboarders do both and have passion for both but chop and change with the conditions. I'm sure most will admit they kite more often.
Bottom line is, have a lesson, see if you like it, if you do, continue, if you don't, stay with the sailboard. And really, don't let other people make up your mind for you, blaze your own trail.

Hope this helps,