The Castle of Lipari sits on the rocky outcrop between the town's two bays, and is the centrepiece of any visit. The Spanish walls you see were built in the 1550s after a devastating Ottoman raid by the corsair Barbarossa; inside them, a tangle of narrow lanes, churches, and small museums occupy the medieval and earlier acropolis.
The most important things on the rock are the Aeolian Archaeological Museum (in several buildings), the Cathedral of San Bartolomeo, and the open archaeological excavation on the south side. Allow at least half a day. Entry to the rock itself is free; the museum charges admission.