Not sure about 'the bay', but if you head 4hrs to the West you'll get good size snapper, whiting, kingys, shark & tuna (when they're about!)

Basically all Vic species, free ramp & 'rarely' any aggro at the boat ramp...(but still a good spectator sport at the ramp due to guys dropping boats on ramp, dodgy brakes etc.)
Or...get on a reputable snapper charter, and stick a small hand-held GPS in your pocket...just don't get caught as you'll end up as berley!

I've actually heard of a couple of charter operators that 'create' their own little secret spots (reefs) using discarded whitegoods, bikes, car parts & the like.