Summer (June–August) is peak season, with the warmest water (often 84°F/29°C) and the longest days. It is also the most crowded and expensive time, and afternoon thunderstorms are nearly daily events — they build fast, dump rain, and usually clear within an hour. Book accommodation months in advance for summer weekends.
Spring Break (March) brings college students in large numbers. The character of the town changes noticeably, and if that's not what you're looking for, avoid the last two weeks of March.
Late spring (May) and early fall (September–October) are the sweet spots — water still warm, crowds significantly thinner, prices lower. September is technically still hurricane season, but the storms that affect this area are relatively infrequent.
Winter (November–February) is quiet, mild, and largely empty. Water temperatures drop too low for most swimmers (low 60s°F/around 17°C), but the beach is beautiful and uncrowded, and you can have St. Andrews State Park almost to yourself. This is when the snowbirds arrive in their campers and stay for months.