Oxford has four distinct seasons and is at its best in April-May and September-November. Spring brings the azaleas and dogwoods in full bloom across the Grove and the historic district; autumn brings cooler weather and the start of the SEC football season, which dominates the town's social calendar from September through November.
Summer (June through August) is hot, humid, and quiet — the university is out, the population drops by a third, and many restaurants run reduced hours. The advantage is that hotel rooms are easier to find and the pace is slow in a pleasant Southern way.
Football season is the busiest time and the most expensive — accommodation on home game weekends regularly sells out months in advance and prices triple. The biggest games (Alabama, Auburn, LSU, the Egg Bowl against Mississippi State at Thanksgiving) are the hardest weekends to find a room. If you are coming for football, book early; if you are not, avoid those weekends.
The Oxford Conference for the Book in the spring and the Oxford Film Festival in February are both worth timing a visit around.