Weymouth Museum is a small local-history museum, but the town gives it good material to work with: harbour trade, Georgian seaside fashion, royal visits, fishing, ferries, and the long habit of turning the beach into a stage. It is the sort of museum that explains why the resort looks the way it does.
Do not expect a major institution. Treat it as a useful, low-key stop while exploring the harbour and old town, especially if bad weather has interrupted a beach day or you want a little context before walking the waterfront.