Atiu is small — about 27 km² — and a scooter is the standard way to get around. Most guesthouses rent them by the day; a full island loop on the perimeter road takes a couple of hours including stops. Bicycles work too if you are content with a slower pace.
For the caves, the marae, and the bush walks you will want a local guide. Marshall Humphreys, Birdman George, and several others run small group tours; book through your accommodation.