Copacabana, on the Bolivian side of the lake, is around three hours from Puno by road including the border crossing at Yunguyo–Kasani. It is a much smaller and more laid-back town than Puno, built around an enormous white pilgrimage basilica that holds a famous 16th-century image of the Virgen de Copacabana — the patron saint of Bolivia.
A day trip is feasible if you start early but better treated as an overnight, especially if you want to continue to the Isla del Sol — the Inca sacred island reachable by boat from Copacabana, half a day's hike across, and one of the most atmospheric places on the lake. Bring your passport, the cost of any visa in cash (USD or BOB), and warm clothes.