May, June, and September are the sweet spots. The water is warm enough to swim, the islands are open, and the crowds — especially the Italian holiday crush — are manageable. Avoid August: prices double, ferries are packed, and everything has to be booked weeks ahead.
July is busy but workable. October to April most hotels and many restaurants close, ferries thin out, and the weather can be windy and wet. Saint Bartholomew's feast (24 August) is the year's biggest local event, with processions and fireworks; worth catching if you are already on the island, not worth fighting the August crowds for.