The old town is best walked slowly, ideally with one of the local elders who act as informal guides. They know which crumbling wall was once a slave depot, which doorway belongs to a family that traces its roots to Oman, and which overgrown corner hides a forgotten grave. Without that context it is easy to walk through and see only picturesque decay.
The architectural mix is genuinely unusual: Swahili carved doors sit alongside German colonial facades and the occasional Indian merchant house. The whole quarter covers no more than a few blocks, but it repays close attention. Look for the oldest doors along the waterfront street — some are a century or more old and still in daily use. The best time to walk is early morning, before the heat builds and while the fishing boats are still coming in.