King's Square is the right place to begin. The town hall, visitor office, harbour, and bridge to Ordnance Island all sit within a few steps, and the square still works as the town's public stage.
In season, historical reenactments can make it feel theatrical, but the setting itself is the point: this is the old civic heart of Bermuda.