Amherst was established in 1807. It began as a simple stagecoach station on the Charlottesville-Lynchburg road, known as “The Oaks” and “Seven Oaks.” The city became the seat of Amherst County after Nelson County was split from Amherst County in 1807. At this time, the town opted to change its name to Jeffery Amherst, 1st Baron Amherst, a French and Indian War hero. Major-General Amherst was a hero of the Battle of Ticonderoga and later served as governor of the Virginia Colony from 1763 to 1768.
Local planter William Waller, who was 58 years old at the time, went from Amherst to Louisiana with roughly 20 slaves for sale in 1847. The Virginia Historical Society has his letters home from the trip, which is a rare record of a slave coffle.
Amherst was incorporated on April 15, 1910, by order of the Circuit Court. In 1950, the General Assembly granted the town a charter.