In order to remain above the water, you have to continually tap A. Simply holding it will result in a gradual decline in altitude, eventually resulting in you landing back in the water. Swimming onto the land cancels the glitch.

Glitch Legend Majora's Mask Swim Trick Zelda