The earliest known settlers were the Siwanoy Indians, an Algonquin tribe. They harvested the rich marshlands for clams and hunted inland for bear, deer, racoon and muskrat. The indigenous population was not long for the area, once the British and Dutch began buying up the land. The next century saw a steady increase in population, as first Quaker refugees from New England and then wealthy New Yorkers established estates in the area. During the late 19th century, Larchmont was known as a summer playground for New York's elite. Many of the large Victorian "cottages" of that era survive in Larchmont Manor. Today, Larchmont is a one-mile-square village within the Town of Mamaroneck, served by the New Haven line of Metro-North Railroad and several major highways.