Granite Island is connected to the Victor Harbor foreshore by a 630-metre timber causeway, and the crossing can be made on foot or aboard the horse-drawn tram that has been running the route since 1894. The island is a granite outcrop rising about 35 metres above the bay, home to a colony of little (fairy) penguins that return to burrows on the island's southern side each evening.
The penguin colony has declined significantly from its peak in the 1990s due to predation by New Zealand fur seals, but guided evening penguin tours still operate and offer the chance to see the birds returning from the sea at dusk. The island is also an excellent whale-watching vantage point from June to October, when southern right whales can be seen in the bay below. The walking trail around the island takes about 45 minutes and has good views back to the mainland and across Encounter Bay.