Carleton was established in 1872 with the arrival of railroad service and named for Carleton B. Emory, son of one of the area's early settlers. Like many of Thayer County's smallest villages, it has seen its population contract significantly since the 20th century.

At 94 residents (2020), Carleton is one of Thayer County's smallest incorporated villages, notable chiefly for its connection to federal mining official Marling J. Ankeny and its proximity to the Prairie Marsh West wildlife area.