Hey Stamp,
This is a tricky one. It depends upon the sailboarder. Was he on the way out or were both of you on the way in.
Sailboarders when on a wave, give way to those coming out through the break, unfortunately it is the opposite to us. When we are on a wave, we have right of way over other kiters on the way out.
Now this is where it gets tricky. If we are on a wave and the poleboarder is on the way out, the rules revert back to us giving way to all other craft in the surf due to our better manoevreability.
If the poley gybed onto the wave, then he was in the wrong. If you have the upwind closest to the breaking section and were already on the wave, then you had the right of way.
Hold your ground but not to the point of having a collision. As Nebbian says, there can be a bit of rivalry between the two camps and some feel it is their god given right to do what they want because their sport was around longer. Yep, there's always one in a crowd

Good winds,