Kaohsiung MRT has two main lines (Red and Orange) that cross at Formosa Boulevard, plus a circular light rail (Yellow) that runs around the inner harbour. It is clean, modern and reaches most things you want to see. Buy an iPass smart card at any station — it works on the MRT, the buses and the ferry to Cijin.
City buses fill the gaps in the MRT network. Routes are well-signed and the iPass works on them too.
Cijin ferry runs every ten minutes from Gushan terminal; the crossing is five minutes and costs less than a US dollar.
Cycling: Kaohsiung has a good network of separated cycle paths along the Love River, around Lotus Pond and on Cijin Island. Youbike rental stations are everywhere; an iPass with a registered phone gets you a bike for the day.
Taxis are cheap and easy to find. Drivers rarely speak English; have your destination written in Chinese.