Koh Phayam is reached by boat from Ranong, the nearest mainland town. Speedboats make the crossing in about 45 minutes; the slow ferry takes around two hours. Ranong itself is a 9-10 hour bus journey from Bangkok (overnight buses are available), or you can fly to Ranong Airport from Bangkok on small aircraft, though schedules are limited.
From the south, Ranong is about 2.5 hours from Chumphon and accessible from Phuket in about 5 hours by road. The boats typically depart in the morning and early afternoon; check times locally as they vary by season and conditions. There are no cars on the island — transport within Koh Phayam is by motorbike (rentable from guesthouses) or longtail boat.