Columbus, Georgia, right across the river, is the obvious first move — the National Infantry Museum at Fort Moore (free, and one of the best military museums in the country), the Columbus Museum (Southern art and decorative arts), the historic district of antebellum and Victorian houses on Broadway, and the bulk of the metropolitan area's restaurants and bars.
Fort Moore (formerly Fort Benning) is the home of the U.S. Army Infantry and Armor Schools. The base itself is restricted but the National Infantry Museum at the entrance is open to the public daily, free, and easily worth half a day.
Callaway Gardens, about an hour's drive east into Georgia near Pine Mountain, is a 2,500-acre resort with botanical gardens, a butterfly conservatory, swimming beaches on lakes, and a long-running summer Florida State Circus performance. Strong choice for families.
For a longer trip, Tuskegee is about 90 minutes northwest — site of the Tuskegee Institute (Booker T. Washington's famous Black agricultural college) and the Tuskegee Airmen National Historic Site at Moton Field, where the African-American fighter pilots of WWII trained.