Walking is the answer. The historic core — Castle Hill, Old Town Square, the main pedestrian streets — is small and compact, and almost everything worth seeing is within fifteen minutes on foot. The Castle Hill itself is a steep climb from the lower town but the road is paved and short.
Local buses connect the centre to the zoo, the arena, and the outer suburbs, and a couple of routes run out to Lake Balaton, but you are unlikely to need them. Taxis are cheap.