Chiltern's centre is the reason to pull off the highway. Main Street and Conness Street keep the scale of a small gold rush town, with verandah-fronted shops, former banks, old hotels, and painted signs clustered close enough to explore on foot.
It is not a blockbuster streetscape like Maldon, but it has a quiet, intact feel. Take the town walk rather than just driving through; the small details, from shop windows to old brickwork, are what make it work.