You can either walk to Acanti on the Colombian side of the border, or take a boat. It's about 5 miles.
There is no road connecting Portobelo to the rest of Panama. There are (irregular) flights to Colon and to Panama City. There are quite a few boats going in the direction of Colon too, but some of the captains are unwilling to take passengers. Boats run by Cuna Indians are the best option - and your best bet to see some of the San Blas islands which is a unique experience.