Kirkwall itself is a small town and entirely walkable — you can get from the harbour to the cathedral in five minutes. The old town is compact enough that you don't need transport to see the main sights.
For the rest of Mainland Orkney, car hire is strongly recommended. The public bus network connects main settlements, but Neolithic sites like Skara Brae, the Ring of Brodgar, and Maeshowe are time-consuming to reach by bus, especially if you want to visit more than one in a day. Car hire is available at Kirkwall Airport and in town.
Cycling is a reasonable alternative: the roads are quiet and the terrain is mostly flat. The route from Kirkwall to the Neolithic World Heritage sites is about 25 km each way. Several shops in town rent bikes.
For visiting the outer islands, small ferries run from Kirkwall pier to most of the inhabited islands. Schedules are available from Orkney Islands Council's ferry service.