The only area where you have to have eyes in the back of your head is on the Northern side of the harbour between Bradleys Head and Middle Head where Ferrys cross and there's a lot of other traffic - particularly at weekends.
But given that thousands of other small dinghies (from Oppies upwards) sail on the harbour it's reasonably OK as long as you keep your wits about you in the shipping channel and try to avoid busy times - like when there's a big race on.
Also when its really busy it gets very choppy and there's no pattern to the waves so it can be unpleasant to sail anyway.
Finding a suitable launching site can be a challenge because while you can launch from a concrete ramp, they're not the easiest to return to. And the boat parks in the sailing clubs are all full or only allow certain classes.
www.australianboatguide.com/NSW-boat-ramps/One other option is to rent a boat from Balmoral Sailing School
www.sailingschool.com.au/I sail a Weta Trimaran (www.wetamarine.com) which I keep "moored" on the road and launch from a pontoon which is accessed by a gantry only 2 metres wide (the Weta is 1.98 metres wide on the trolley before assembly). It's a bit of a challenge but it's only 5 mins from home on the bike. You can sail it single handed or with up to 3 adults and it will do up to 20 knots off the breeze (my max is 16.7 knots in the harbour).
My tip: Always wear bright clothing and a life jacket!