Powysland Museum, now part of Y Lanfa by the canal wharf, is a small local museum covering the archaeology and social history of old Montgomeryshire. The restored warehouse setting helps: you are right beside the canal, in the part of Welshpool where trade and transport shaped the town.
Do not expect a major museum. Expect a useful hour that gives Powis Castle and the surrounding countryside some local context, especially if rain has pushed you indoors.