Hunter Island was originally inhabited by the Siwanoy people, a subgroup of the Lenape Native Americans. The island was called "Lap-Haa-Waach King," meaning "place of stringing beads," reflecting its role in local indigenous shell bead-making and ceremonial activities. On 23 June 1654, English physician Thomas Pell purchased the island along with large surrounding lands from the Siwanoy chiefs, formalizing European claims. Early colonial names for the island included Pell's Island, Pelican Island, and Appleby’s Island, reflecting various private owners prior to the Hunter family acquisition.