The incident happened at the Tarrytown Reservoir just before 11 a.m.
Police say the driver managed to escape the sinking rig before it was submerged.
He was rushed to the hospital by Greenburgh EMS with symptoms of hypothermia, and was treated and released.
Tarrytown Police say the Recreation Department was plowing snow off of the reservoir when it went through the ice. His identity has not been released.
The tractor was removed from the water after the incident.