Hey 'slave,
I just checked out Google Earth and it is about 100km as the crow flies from Port. It will take you about 2 hours cause you have to go up through the hills and through the inland way. You don't need a 4WD but that could also depend on if it has been raining heaps.
We went there in the old bus and had no problems at all. 50% of the road was tar and 50% was graded dirt but that was back in 2003, and I suspect the road would have been finished by now all the way to Cooktown.
We absolutely loved Cooktown as a great place to visit and was one of the friendliest and and best places we stayed in our whole trip around Oz. We stayed in the caravan Park on the edge of town as you first drive in. It's only a 10 minute drive to the turnoff to Archer Point from the Cara park and then another 20 minutes or so down the dirt track to Archer Point.
There was nothing at all at the Point so anything you need, you need to bring in with you. Great spot to camp, grassy right next to the beach.
Just be a little wary of the logs with eyes but aside from that, you'll be right. Like I said, windiest place in QLD. I was lit on my 11.5 big time when we visited.
Your other alternative to hang with Bretto from Windswell is also a good idea, but take the reef trip cause the wind in Port usually sucks

Have a good trip.
Good winds,