The caves, lake, Bhutanatha temples, and fort area are close enough to walk, though the climbs are uneven and summer heat is serious. Wear shoes with grip; polished sandstone steps can be slippery.
For Mahakuta, Banashankari, Aihole, and Pattadakal, use a hired car, auto-rickshaw by negotiation, or a local bus if you have time and patience. A car makes the temple circuit much less tiring.