Castro's Plaza de Armas is the city's natural centre, framed by shops, civic buildings, and the yellow towers of the Church of San Francisco. It is not grand, but it is the place where the town's errands, bus connections, and sightseeing all overlap.
Use it as your orientation point. From here it is a short walk downhill to the waterfront and a short detour to the Municipal Museum.