Central Tsuyama works well on foot. Kakuzan Park is roughly 15 to 20 minutes from the station, and Joto Street and Shurakuen Garden are reachable with a longer walk.
Bike rental is a useful option if you want to cover the old streets, garden, railway museum, and outer shrines without watching bus times.