Chiayi Municipal Museum is the general city museum, useful for tying together local history, art, natural history, and the Alishan hinterland. It is close enough to Hinoki Village and the railway park to include without making a separate trip.
Do not expect a blockbuster museum. Its value is orientation: it helps explain why Chiayi looks toward the mountains, why cypress timber matters here, and how the city fits into southern Taiwan.