Fort St. Catherine anchors the military side of St. George's. Its position above the water makes the defensive logic obvious: ships approaching Bermuda's east end had to reckon with these guns and walls.
Pair it with the town rather than treating it as a separate excursion. The walk from narrow lanes to open sea is part of the pleasure.