Police say Timothy Miller was kayaking when he flipped the boat and lost a paddle Saturday evening.
He swam to a small island and rested before trying to swim to shore but the current was too strong.
He called 911 with his cell phone and held onto the pylon while he waited for help.
Suffolk County police say Marine Bureau Officers Michael Axelson, Frank Sierra and Brendan Grogan of Marine Mike responded. A Suffolk County helicopter guided the boats because of the shallow water.
The officers pulled the 39-year-old sportsman onto the boat where he was taken to a hospital as a precaution.