I believe locals wavesail at middle rock at Lake Cathie ('cat eye') south of Port. On my road trips north I normally sail (southerly) elsewhere and skip Port as it's off the highway.
I think some of the surfing spots are slightly over-rated there but had some good surfs at Town's which likes swell from NE-SE, it is popular and could be considered a bit crowded.
Unfortunately the river is infamous for bull sharks.
You could send PM to AJEaster for local knowledge.