The old town is small and flat enough to walk. From the railway station it is a 15-minute walk north to the Grote Kerk, and another 10 to the Groothoofdspoort. Bicycles are everywhere; the OV-Fiets rental at the station is the cheapest and most reliable option and saves you the walk back.
The waterbus is the most enjoyable bit of local transport — fast catamarans linking Dordrecht with Rotterdam, Papendrecht, Zwijndrecht and a series of waterside stops along the way. It runs on a regular schedule, takes bikes free, and gives you a closer view of the river traffic than any train will. Buses run within the city but are rarely needed unless you are staying out in the southern suburbs.