TheGeneral said..
Have you looked into waterproof cases, such as Aquapac? I don't carry a phone while kiting, but I keep my car key in a small Aquapac that I tuck inside my rashie/wetsuit. Been doing that for two seasons and haven't had a problem. Not cheap, but a quality product, and I'd definitely trust a splash-resistant phone to it, unlike the $5 triple-ziplock case I got in Thailand.
If you got a call or text, it might be a hassle to get your phone out. You could Bluetooth it to one of the latest GPS watches, such as a Suunto Ambit 3 or Garmin Fenix 2, which would give you a great excuse to get one, if you haven't already!
I looked at waterproof arm pouches and none seemed big enough to hold the S5 with the flip cover on it. I'm thinking of maybe a very small waterproof back pack, but then that would make it hard to get to the phone?