High Island (originally named High Isle) acquired its name from the hilltop in the center of the island which is the highest point within the Muscle Ridge chain of islands. Until late in the 19th century, High Island was mostly uninhabited except for occasional seasonal use by fishermen or a fishing family.
In the early part of the twentieth-century, granite mining operations began as an adjunct to the quarry on nearby Dix Island. A residential village and quarry operations, buildings and structures were built on High Island to accommodate the needs of the 500 to 600 workers that inhabited the island during the height of granite harvesting activities.
After the mining operations and periodic post-mining use of the buildings ceased, circa 1914, the island was once again abandoned.
Some of the unique features on the island include three water-filled quarry pits. The largest quarry is on the southwest end of the island and is approximately 0.63 acres in size and quite spectacular in both size and beauty.
The quarry is located amidst a dense vegetative cover of native and naturalized vegetation wrapping around high ledges. Views from above the quarry extend south in a 200-degree arc over the other islands within Muscle Ridge Shoals. There are three historic wharfs located to the south of the historic village remains of the island.